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ABSTRACT 


Health  care  professionals  are  confronted  daily  with  a  deluge  of  new 
medical  information.  The  ability  to  deliver  standardized,  high  quality 
health  care  is  based  on  the  clinician’s  initiative  to  seek  and  evaluate 
this  new  information.  In  addition,  human  memory  is  often  imperfect  and 
is  further  restricted  by  fatigue,  stress  and  a  demanding  work  schedule. 

A  computerized  bedside  clinical  information  system,  the  CLINICAL 
REFERENCE  LIBRARY,  was  developed  to  facilitate  the  management  of  patient 
care  and  to  provide  physicians-in-training  and  paramedical  personnel 
with  relevant  clinical  information  about  common  disease  processes  and 
problems  encountered  in  the  intensive  care  unit. 

The  CLINICAL  REFERENCE  LIBRARY,  developed  and  described  in  this 
thesis,  is  a  subsystem  of  the  HP  5600A  Patient  Data  Management  System. 

It  contains  20  information  modules.  Each  module  consists  of  several 
individual  computer  displays.  Where  applicable,  information  basic  for 
any  single  problem  is  classified  under  the  following  headings: 
definition,  incidence,  etiology,  bacteriology,  pathology, 
pathophysiology,  clinical  course,  diagnosis,  treatment,  complications, 
prognosis,  and  references.  Topics  include  medical  record  keeping, 
airway  management,  pulmonary  physiology,  acid-base  regulation, 
acid-base  abnormalities,  gas  transport,  pulmonary  embolism.  Adult 
Respiratory  Distress  Syndrome,  septic  shock,  aspiration  pneumonia, 
poisonings,  antimicrobial  therapy,  adverse  drug  interactions,  pediatric 
medications,  nutritional  support,  burns,  hemodynamic  monitoring,  cardiac 
arrhythmias,  brain  resuscitation  and  hypothermia. 

The  system  is  menu-driven  and  self-instructing.  The  user  is  able  to 
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flip  rapidly  back  and  forth  through  any  module,  or  from  one  module  to 
another,  much  as  one  would  flip  through  a  book.  Logging  and  analysis  of 
user  performance  are  also  incorporated  into  the  CLINICAL  REFERENCE 
LIBRARY. 

Evaluation  of  the  CLINICAL  REFERENCE  LIBRARY  was  in  the  form  of  a 
classical  non-randomized ,  pre-test  -  post-test,  control-experimental 
research  paradigm.  Three  specific  findings  emerged  from  the  data: 

1)  medical  students  and  residents  exposed  to  the  CLINICAL 
REFERENCE  LIBRARY  "acquired"  mohe  knowledge  than  their 
counterparts  who  used  conventional  teaching  methods 

2)  medical  students  and  nurses  accessed  the  CLINICAL  REFERENCE 
LIBRARY  for  continuing  medical  education  whereas  residents  used 
the  system  primarily  for  problem-solving,  and 

3)  a  general  attitudinal  ambivalence  was  directed  toward  the 
CLINICAL  REFERENCE  LIBRARY  by  the  nurses. 
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The  scientific  work  of  antiquity  was  disseminated  by  word  of  mouth, 
by  correspondence  or  by  laborious  copying  of  manuscripts  by  scribes. 
Scholarly  activities  were  restricted  to  a  few  centers  where  extensive 
collections  of  manuscripts  were  available.  The  development  of  the 
printing  press  in  the  fifteenth  century  revolutionized  the  dissemination 
of  knowledge.  Today,  the  computer  and  its  application  in  information 
storage  and  retrieval  and  data  acquisition  and  analysis  has  created  an 
equal  or  perhaps  a  greater  revolution  than  the  invention  of  the  printing 
press  (3*0. 

THE  KNOWLEDGE  EXPLOSION 

During  the  past  50  years,  there  have  been  rapid  advances  in 
medicine  and  its  allied  sciences.  Consequently,  there  has  been  a 
phenomenal  increase  in  technical  and  scientific  knowledge  (52).  The 
1933  edition  of  Cecil's  Textbook  of  Medicine  is  one  such  illustration  of 
the  progress  made  in  the  last  50  years  (12).  There  was  no  effective 
treatment  for  bacterial  mycobacterial  or  fungal  diseases;  thus, 
bacterial  endocarditis,  miliary  tuberculosis  and  cryptococcal  meningitis 
were  fatal  diseases.  Adrenal  steroids  had  not  been  discovered. 

Diuretics  were  not  available  for  the  management  of  heart  failure,  nor 
were  there  any  drugs  for  the  control  of  hypertension.  Anticoagulants 
had  not  yet  been  introduced.  The  role  of  lymphocytes,  immune  complexes 
and  complement  had  not  been  elucidated.  Discussion  of  "acidosis"  and 
"alkalosis"  were  based  on  only  one  clinical  test  -  i.e.  measurement  of 
the  carbon  dioxide  combining  power  of  the  blood.  Surgical  removal  of 
tonsils  and  adenoids  were  ethusiastically  recommended:  "The  beneficial 
results ..  .are  so  great  and  so  prompt  that  unless  there  is  a  distinct 
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reason  for  its  omission,  it  is  folly  to  wait  for  spontaneous  shrinking 
...”  (12).  Mechanical  ventilation,  renal  dialysis,  extracorporeal 
membrane  oxygenation,  cardioversion,  and  cardiac  pacing  were  unknown. 
Bleeding  peptic  ulcer  was  managed  by  the  application  of  an  icepack  to 
the  epigastrium,  complete  starvation,  morphine  hypodermically,  and 
water,  glucose  and  soda  by  rectal  drip.  Blood  banks  were  not  available 
and  the  transfusion  of  a  single  unit  of  blood  was  a  formidable  pro¬ 
cedure.  Diagnostic  procedures  in  clinical  biochemistry  and  immunology, 
which  are  now  commonplace,  had  not  yet  come  into  use.  Investigative  and 
diagnostic  procedures  such  as  angiography,  sonography,  computerized 
tomography  and  fiberoptic  endoscopy  were  unknown. 

More  recently,  medicine  has  seen  "cures”  in  childhood  leukemia, 
computerized  tomography,  mass  production  of  insulin  and  growth  hormone 
using  recombinant  DNA  technology,  in  utero  surgical  repair  of  congenital 
anomalies,  laser  microsurgery,  radionuclide  imaging,  reimplantation  of 
traumatically  amputated  extremities  and  digits,  the  development  of 
artificial  skin  for  burn  victims  and  the  transplantation  of  body  organs 
(corneas,  kidneys,  heart,  bone). 

Because  medical  knowledge  is  continually  changing  and  expanding, 
the  number  of  medical  problems  encountered  are  also  increasing.  The 
reasons  for  this  are  many:  surgical  intervention  in  the  very  young  and 
the  elderly,  the  debilitated  and  poor-risk  patients;  the  use  of 
steroids,  chemotherapy  and  irradiation  predispose  patients  to 
complications  and  disease  processes  that  were  unknown  half  a  century 
ago;  the  widespread  use  of  antimicrobials  and  the  common  practice  of 
polypharmacy  are  not  without  adverse  effects.  Mass  casualties  and 
multiple  trauma  are  common  events  in  today’s  society.  Respiratory 
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failure,  septic  shock  and  chronic  renal  failure  are  therefore  diseases 
of  modern  medicine. 

During  the  past  decade,  advances  in  medicine  have  lead  to  a  deluge 
of  information  and  technology,  making  the  contempory  medical  care  system 
more  complex,  more  information-dependent,  and  more  technology-oriented. 
The  critically  ill  or  injured  patient  also  requires  more  specialized 
care.  Changes  in  the  concepts  and  methods  of  care  of  these  patients 
reflect  not  only  the  development  of  new  therapeutic  measures  and  the 
understanding  of  various  disease  processes,  but  also  a  more  aggressive 
attitude  by  physicians  toward  therapy  and  the  availability  of  equipment 
for  monitoring  and  resuscitation.  This  combination  of  new  attitudes, 
methods,  and  equipment  is  exemplified  by  the  intensive  care  unit  (ICU) 
and  by  specialty  units  in  tertiary  care  centers.  Regardless  of  how  a 
patient  gets  into  serious  life-threatening  difficulty,  every  critically 
ill  patient  enters  a  "final  common  pathway"  of  multisystem  failure. 
Consequently,  multidisciplinary  teams  -  including  medical  and  surgical 
specialists,  anethestists ,  nurse  specialists,  and  allied  technical 
personnel  -  are  required  to  care  for  the  critically  ill. 

KNOWLEDGE  ACQUISITION  IN  MEDICINE 

To  the  practicing  clinician,  the  acquisition  of  medical  knowledge 
is  a  life-long  process  which  includes  not  only  the  formal  education  at 
the  undergraduate  and  postgraduate  (residency)  medical  training  levels, 
but  also  the  more  informal  education  accompanying  active  patient  care. 
The  amount  of  knowledge  that  is  needed  to  address  all  medical  problems 
has  far  surpassed  the  ability  of  any  one  physician  to  consistently 
retain,  recall  and  apply  this  information  (6).  Although  this  informa- 
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tion  exists  in  various  forms  (textbooks,  journals,  informal  discussions, 
etc),  it  is  usually  not  available  in  a  manner  readily  accessible  to  the 
physician . 

How  then  do  physicians  keep  abreast  of  medical  information? 
Continuing  medical  education  (CME),  initially  implemented  on  a  voluntary 
basis,  is  now  a  requirement  for  reregistration  by  many  state  medical 
licensing  boards  and  for  recertification  by  some  specialty  boards  in  the 
United  States  (61).  Mandatory  continuing  medical  education  programs  do 
not  exist  in  Canada  (5).  CME  and  assessment  can  be  in  the  form  of 
accredited  post-graduate  courses  and  workshops  (41),  computer-based 
education  (CBE)  (49),  and  home  study  CME  courses  (47,  65). 

Scientific  congresses,  informal  personal  conversations, 
professional  consultation,  rounds,  seminars  and  journal  clubs  are 
valuable  adjuvants  to  the  educational  process.  Symposia  and  discussions 
are  sometimes  tape-  or  video-recorded  and  subsequently  published  e.g. 
Audio-Digest  and  Medifacts.  The  circulation  of  manuscripts  within  the 
academic  environment  is  a  method  whereby  interested  colleagues  have 
requested  that  they  "be  kept  up  to  date  with  developments  in  your  lab". 
Much  useful  information  is  conveyed  by  leaflets,  brochures,  and 
newsletters  produced  by  pharmaceutical  manufacturers,  suppliers  and 
non-profit  organizations  e.g.  MEDICAL  LETTER,  CLINICAL  SYMPOSIA  and 
CLINICAL  CONSULTATIONS. 

Textbooks,  journals  and  periodicals  are  the  most  frequently  cited 
means  of  obtaining  information  to  satisfy  a  particular  need,  whether  it 
be  for  problem-solving,  continuing  medical  education  or  recreation  (46). 
Library  services  include  the  borrowing  of  books,  audio-visual  materials, 
telephone  enquiry  services  (9)  and  photocopying  services.  Indexing 
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services  such  as  INDEX  MEDICUS  and  CITATION  INDEX  are  printed  monthly 
and  updated  annually. 

Computerized  information  networks  such  as  MEDLARS  (Medical 
Literature  Analysis  and  Retrieval  System)  and  MEDLINE  (MEDLARS  On-Line) 
are  the  most  complex  and  extensive  medical  information  retrieval  systems 
(39). 

The  problem  of  keeping  abreast  of  new  medical  information  is  even 
more  pronounced  in  the  intensive  care  unit  (ICU)  as  exemplified  in  the 
following  section. 

MEDICAL  INFORMATION  IN  THE  INTENSIVE  CARE  UNIT 

For  the  purpose  of  clarity,  a  brief  definition  of  some  terms  with 
respect  to  patient  management  is  in  order.  In  medicine  many  terms  are 
used  interchangeably  to  describe  the  cognitive  process. 

"Problem-solving"  is  the  process  of  developing  a  differential  diagnosis 
and  ultimately  arriving  at  a  final  working  diagnosis  and  eventually  a 
correct  diagnosis.  The  term  "medical  inquiry"  focuses  on  the 
data-gathering  or  evaluative  aspect  of  this  process.  The  terms 
"clinical  judgment"  or  "medical  decision-making"  focus  on  the 
decision-making  component.  For  these  reasons,  "clinical  reasoning"  has 
been  chosen  as  the  term  to  encompass  all  the  cognitive  skills  implied  in 
patient  evaluation  and  management  (4). 

At  the  present  time,  a  clinician  working  in  the  intensive  care  unit 
uses  a  combination  of  intuitive  skills  and  knowledge  acquired  through 
experience.  In  times  of  life-threatening  crises  the  traditional 
sequence  of  detailed  enquiry  for  diagnosis,  investigation,  confirmation, 
and  therapy  is  unsatisfactory  (4).  (Indeed,  even  in  nonlife-threatening 
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situations  it  is  unsatisfactory.  There  is  an  incongruence  between  how 
people  are  taught  and/or  trained  to  resolve  problems  and  how  it  is 
actually  done  by  practicing  physicians.  But  this  process  is  even  more 
important  in  the  intensive  care  unit.)  Priorities  must  be  established 
and  therapy  must  be  set  into  motion  if  life-threatening  and  life-saving 
measures  are  to  be  effective.  Thus,  clinical  reasoning  and  therapeutic 
intervention  in  the  intensive  care  unit  proceed  in  parallel  rather  than 
in  followup  to  a  diagnostic  study. 

Critically  ill  patients  often  have  multiple  problems  on  admission 
and  during  their  stay  in  the  intensive  care  unit  may  develop  one  or  more 
problems.  This  is  illustrated  in  the  following  typical  case  history. 

A  38  year  old,  morbidly  obese  female  underwent 
jejunoileal  bypass  and  was  lost  to  followup.  Progressive 
and  unremitting  weight  loss  ensued  and  at  eleven  months  her 
weight  had  fallen  from  193  to  97  kg.  She  was  admitted  to 
hospital  complaining  of  lethargy  and  circumoral  paresthesia. 
Laboratory  investigation  revealed  a  markedly  depleted  serum 
albumin,  hypocalcemia,  hypokalemia  and  a  depressed  total 
lymphocyte  count.  The  surgeon  elected  to  interpose  an 
additional  15  cm  of  ileum.  The  patient  was  discharged  on 
the  fifth  postoperative  day  after  an  "unremarkable"  course 
despite  a  low  grade  fever  which  was  attributed  to  mild 
atelectasis . 

Three  days  later,  she  was  readmitted  with  severe 
abdominal  pain,  nausea  and  vomiting.  She  was  afebrile. 

Radiographs  revealed  marked  gastric  dilatation.  Laboratory 
investigation  revealed  a  moderate  leukocytosis  with  a  "shift 
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to  the  left".  She  was  initially  treated  with  nasogastric 
suction  and  intravenous  fluids  but  the  nasogastric  tube  was 
removed  on  the  third  day  and  her  diet  advanced.  On  the 
seventh  day,  the  patient  suddenly  developed  profound 
hypotension  and  a  rigid  abdomen. 

After  the  administration  of  large  volumes  of 
intravenous  fluids,  steroids  and  antibiotics  she  underwent 
an  emergency  laparotomy  which  revealed  a  disrupted 
anastomosis  and  extensive  intraabdominal  soilage. 

Peritoneal  lavage  was  performed  and  an  ileostomy  and 
mucocutaneous  fistual  were  created.  She  was  then 
transferred  to  the  intensive  care  unit. 

Despite  aggressive  management,  the  patient  went  on  to 
develop  respiratory  failure,  renal  failure,  gastrointestinal 
hemorrhage,  wound  dehiscence  and  systemic  candidemia.  She 
died  on  the  fourtieth  postoperative  day. 

In  summary,  this  patient  presented  with  signs  and 
symptoms  of  malnutrition  following  an  intestinal  bypass 
procedure.  Had  this  patient's  markedly  malnourished  state 
been  corrected  before  the  elective  operation  the  subsequent 
events  could  well  have  been  avoided.  Because  of  a  delay  in 
the  diagnosis  of  intraabdominal  sepsis,  she  developed 
bacteremic  shock  and  its  sequelae.  Intensive  postoperative 
therapy  with  wide  spectrum  antibiotics,  mechanical 
ventilation,  hemodialysis  and  hyperalimentation  were 
un successful . 

This  example  illustrates  the  need  for  adequate 
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information  in  the  ICU:  What  is  the  course  of  this  disease?  What  are 
the  complications?  What  is  the  optimal  form  of  therapy?  What 
therapeutic  options  are  contraindicated  and  why?  What  are  the 
complications  of  therapy  and  how  can  they  be  prevented?  What  is  the 
patient's  prognosis?  Can  the  prognosis  be  altered  and,  if  so,  how? 

The  health  care  professional  may  or  may  not  have  this  information. 
Information  sources  in  the  intensive  care  unit  are  variable.  Human 
resources  beyond  that  of  the  resident-on-call  are  generally  limited  to 
the  chief  resident  and/or  attending  staff.  The  supervisor  may  not  be 
readily  available.  Fellow  residents,  nurses  and  other  paramedical 
personnel  are  limited  sources  of  information.  Textbooks  are  costly, 
easily  misplaced  and,  if  succinct,  readable,  and  portable,  are  often 
misappropriated.  The  hospital  library  is  not  usually  in  close  proximity 
to  the  intensive  unit  and  even  if  it  is  accessible,  there  is  no 
guarantee  that  the  information  will  be  available.  Thus,  the  major 
drawback  is  that  there  is  no  one  up-to-date,  clinically  relevant 
information  source  that  is  available  around  the  clock  and  located  in 
close  proximity  to  the  bedside. 

No  one  physician  knows  all  the  information  necessary  to  practise  all 
aspects  of  medicine.  Human  memory  is  often  imperfect  and  is  further 
restricted  by  fatigue,  stress  and  a  demanding  work  schedule.  Attempts 
to  overcome  these  deficiencies  have  lead  to  the  development  of 
computer-based  consultation  systems.  In  theory,  these  systems  have 
rapid,  accurate  and  infinite  processing  capabilities,  provide  an 
immediate  source  of  information,  are  availablle  24  hours-a-day,  are 
located  physically  within  the  intensive  care  unit,  and  serve  to 
reinforce  concepts  and  principles  of  patient  management  and/or  disease 
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processes.  Unfortunately,  most  systems  developed  to  date  are  not 
ICU-directed  or  -oriented. 

STATEMENT  OF  THE  PROBLEM 

In  an  effort  to  provide  the  health  care  professionals  in  the 
intensive  care  unit  with  readily  available  and  systematic  information,  a 
computer-based  clinical  information  system  (CLINICAL  REFERENCE  LIBRARY) 
was  designed.  This  bedside  system  contains  information  about  common 
problems  encountered  in  the  ICU:  the  criteria  for  intubation  and 
assisted  ventilation;  the  complications  of  a  tracheostomy;  the 
pathophysiology  of  acid-base  abnormalities;  the  management  of  a  patient 
with  a  pulmonary  embolus,  or  aspiration  pneumonia  or  spectic  shock;  drug 
interaction;  pediatric  medications;  the  rational  use  of  antimicrobials; 
the  management  of  the  head-injured  patient;  the  insertion  of  a  Swan-Ganz 
catheter  and  the  interpretation  of  the  various  wave  forms;  the 
pathophysiology  of  starvation  and  the  implementation  of 
hyperalimentation;  and  the  interpretation  and  treatment  of  cardiac 
arrhythmias . 

Two  specific  questions  were  asked: 

1)  Would  this  bedside  information  system  be  used  by  clinicians, 
graduate  and  undergraduate  physicians-in-training ,  nurses,  and 
other  paramedical  personnel? 

2)  Would  this  computer-based  clinical  information  system  be  an 
effective  teaching  tool? 

AN  OVERVIEW  OF  THE  CONTENTS 


The  present  thesis  describes  the  development  and  evaluation  of  a 


computer-based  clinical  information  system  in  the  intensive  care  unit. 

In  this  chapter,  the  conceptual  need  for  the  development  of  such  a 
system  is  discussed.  The  application  of  computers  in  medicine  and  the 
use  of  computer-based  medical  information  systems  are  described  in 
Chapter  II.  This  is  followed  by  a  description  of  the  intensive  care 
delivery  system  at  the  University  of  Alberta  in  Chapter  III  and  the 
description  of  the  HP  5600A  COMPUTERIZED  PATIENT  DATA  MANAGEMENT  SYSTEM 
in  Chapter  IV.  Chapter  V  provides  a  detailed  summary  of  the  development 
of  the  CLINICAL  REFERENCE  LIBRARY.  In  Chapter  VI  the  methodology  of 
evaluation  is  described.  The  analysis  and  results  of  participant  data 
and  the  results  of  questionaires  about  the  CLINICAL  REFERENCE  LIBRARY 
are  presented  in  Chapter  VII.  Discussion  of  the  data  and 
recommendations  for  further  research  and  development  are  presented  in 
Chapter  VIII. 


CHAPTER  II 

REVIEW  OF  THE  LITERATURE 
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In  this  chapter,  literature  related  to  the  role  of  computers  in 
medicine  and  clinical  medical  information  science  will  be  presented. 

The  chapter  is  divided  into  five  parts.  The  first  part  is  a  brief 
introduction  about  the  history  of  computers.  This  is  followed  in  part 
two  with  a  summary  of  the  application  of  computers  in  medicine.  A 
review  of  computer-based  information  systems  is  presented  in  part  three. 
The  impact  of  computers  on  clinical  medical  information  science  is 
presented  in  part  four.  The  chapter  is  summarized  in  part  five. 

INTRODUCTION 

The  first  "computer"  devices,  the  oriental  abacus  and  the 
occidental  Antikythera  device,  were  in  use  over  2000  years  ago.  The 
development  of  computers  lay  dormant  until  Pascal's  invention  of  a  small 
mechanical  adding  machine  in  the  seventeenth  century.  The  first 
operational  automatic  computer  was  introduced  in  1944.  Digital  com¬ 
puters  were  virtually  unknown  in  medicine  in  the  1950's  (22). 

THE  ROLE  OF  COMPUTERS  IN  MEDICINE 

Progressive  improvements  in  the  manufacturing  of  integrated 
circuits  has  reduced  the  size  and  cost  of  many  electronic  devices.  The 
relative  inexpensiveness  and  miniturization  of  computers  have  led  to  the 
development  of  many  new  medical  technologies.  One  notable  example  is 
the  computerized  tomography  scanner  which  produces  precise  images  of 
anatomical  cross-sections  with  no  discomfort  to  the  patient,  thereby 
dramatically  reducing  the  use  of  invasive  techniques. 
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Computer-Assisted  Data  Acquisition 

Computers  have  been  programmed  to  record  and  analyze  electro¬ 
cardiograms  (EKGs)  in  real-time  (51),  to  assist  in  cardiac  catheter¬ 
ization  (35),  to  take  medical  histories  and  to  enter  data  in  a  variety 
of  clinical  settings  such  as  radiology  (53),  intensive  care  medicine 
(73),  obstetrics  and  gynecology  (23),  and  ophthalmology  (42). 

Computer-Assisted  Diagnosis 

Automated  systems  can  assist  a  physician  in  medical  diagnosis  (8). 
This  is  of  particular  value  where  speed  of  evaluation  is  crucial,  as  in 
the  management  of  the  critically  ill  patient.  In  such  applications, 
computers  offer  easy  access  to  large  quantities  of  medical  information 
and  an  ability  to  analyze  multiple  variables  simultaneously. 
Computer-assisted  diagnosis  has  been  successfully  applied  in  the 
analysis  of  EKG  arrhythmias  (51),  vectorcardiography  (1),  and 
electrolyte  and  acid-base  disorders  (8). 

Computers  and  Medical  Therapy 

Computer-based  devices  can  assist  physicians  in  decisions  relating 
to  therapy:  automated  infusion  pumps  (7),  measurement  of  blood  volume, 
urine  flow  and  blood  lactate  (70),  hemodialysis  (56),  drug  interaction 
(3)  and  neurostimulation  (32). 

Monitoring/Control  of  Physiological  Systems 

Monitoring  devices  presently  found  in  the  intensive  care  unit 
activate  alarms  to  summon  the  medical  staff  when  a  patient  needs  special 
attention.  Devices  have  been  developed  for  in-hospital  control  of  blood 
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pressure  (30),  and  fluid  resuscitation  and  control  of  ventilation  using 
closed-loop  (feedback)  technology  (71).  At  present,  the  development  of 
closed-loop,  implantable  devices  for  long-term  use  in  control  of  blood 
pressure  or  blood  sugar  is  hampered  by  a  lack  of  miniturized  sensors 
capable  of  functioning  in  vivo  for  extended  periods  of  time  (21). 

A  variety  of  microprocessor-based  monitors  are  available  which  do 
not  attempt  to  control  the  patient’s  physiology.  Hospital-based  systems 
include  fetal  (36)  and  obstetrical  monitoring  (23),  and  blood  pressure, 
EKG  and  respiratory  monitoring  during  anesthesia  (27).  Cardiovascular 
telemetry  (38)  and  EKG  monitoring  (69)  are  examples  of  out-patient 
monitoring . 

Computers  and  the  Handicapped 

Microprocessor-based  prostheses  can  control  crude  motorized 
artificial  hands  by  analyzing  myoelectric  signals  arising  from  forearm 
stumps  (24).  Microcomputers  have  been  used  in  the  development  of 
hearing  aids  and  sound  perception  (20),  synthetic  voice  (28), 
voice-controlled  wheel  chairs  (13),  and  writing  aids  for  the  motor 
handicapped  (50) . 

Computers  and  Hospital  Information  Systems 

Computers  have  found  application  in  the  management  of  patient 
information  in  physicians'  offices  (10),  emergency  departments  (2),  and 
out-patient  departments  (44).  CLINFO  (67),  COSTAR  (57),  and  COMTRAC 
(37)  are  examples  of  successful  hospital  laboratory  and  administration 


information  systems. 
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Computers  in  Medical  Education  and  Assessment 

Computers  are  widely  used  throughout  the  continuum  of  medical 
education  and  assessment  (14).  It  has  been  stated  that  they  allow  a 
much  wider  range  of  individualized  responses  in  tutorial  sessions,  a 
greater  degree  of  ’’instructor"  responsiveness  to  individual  background 
and  needs,  and  more  immediate  feedback  in  the  teaching  or  training 
purposes.  Computers  also  provide  a  more  objective  testing  mechanism  for 
assessing  clinical  problem-solving  abilities  (14). 

TICCIT  (Time-Shared  Interactive  Computer-Controlled  Information) 
and  PLATO  (Programmed  Logic  for  Automated  Teaching  Operation)  are 
exclusive  educational  systems  (14).  The  latter  is  more  popular.  PLATO 
provides  interactive,  self-paced  instruction  for  large  numbers  of 
students.  Lesson  material  is  presented  on  a  video  display  terminal  and 
may  consist  of  text,  drawings,  graphs  and  color  photographs.  Students 
interact  with  the  material  through  a  special  keyboard  that  closely 
resembles  a  typewriter  keyboard.  Students  receive  instruction  re¬ 
inforcement  of  correct  work  and  assistance  where  they  are  having 
difficulty.  The  users  of  PLATO  range  from  grade  school  students 
learning  reading  and  mathematics  to  graduate  students  in  the  medical 
sciences . 

The  Faculty  of  Medicine  at  the  University  of  Alberta  has  conducted 
undergraduate  teaching  of  cardiology  using  PLATO.  Instruction  is  given 
in  heart  sounds  and  murmurs,  electrocardiography,  common  valvular 
lesions  and  common  congenital  problems.  There  are  a  number  of  test 
cases  and  examinations  have  been  conducted  to  determine  user  performance 
(33). 

The  College  of  Medicine  at  Ohio  State  University  employs  an 
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optional  independent  study  program  (ISP)  for  completing  the  basic 
science  portion  of  the  Doctor  of  Medicine  curriculum.  In  contrast  to 
the  traditional  lecture-discussion  program  in  which  students  progress 
through  a  standard  content  sequence  in  a  predetermined,  fixed  amount  of 
time,  the  independent  study  program  follows  a  self-study  format  which 
allows  progress  at  independent  rates  (14). 

Several  different  models  of  computer-based  simulations  are  used  for 
educational  assessment.  The  Royal  College  of  Physicians  and  Surgeons  of 
Canada  has  used  Computerized  Patient  Management  Problems  (CPMPs)  in 
their  Pediatric  Fellowship  Examination  process  to  test  the  ability  of 
pediatric  candidates  to  manage  patients  (63,  66).  CPMPs  were  also  used 
between  1974-79  as  part  of  the  examination  process  for  approximately  110 
final  year  medical  undergraduates  at  the  University  of  Alberta  (14). 
CPMPs  have  been  incorporated  into  a  self-assessment  package  that  is 
available  to  fellows  of  the  Royal  College  during  the  time  of  the 
College's  annual  meeting  (14). 

The  Computer-Based  Examination  (CBX)  Project  of  the  American  Board 
of  Internal  Medicine  and  the  National  Board  of  Medical  Examiners  is 
based  on  a  computer  simulation  of  a  patient  with  a  given  disease  or 
diseases  (26,  58).  Laboratory  tests,  medical  procedures  or  consult¬ 
ations,  and  drug  therapies  interact  with  the  patient's  disease  in  a  time 
sequence  similar  to  one  occuring  in  real  life.  By  simulating  the  effect 
of  physician  action  on  the  patient,  and  by  adjusting  the  patient's 
status  accordingly,  the  computer  model  provides  an  almost  life-like, 
dynamic  simulation  of  the  patient-physician  encounter. 

The  MERIT  (Model  for  Evaluation  and  Recertf ication  through  In¬ 
dividualized  Testing)  system  was  designed  to  incorporate  individualized 
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evaluations  of  a  candidate's  performance  in  patient  care  as  a  major  part 
of  the  recertification  procedure  for  the  American  Board  of  Internal 
Medicine  (14).  A  computer-assisted  simulation  of  the  clinical  patient 
encounter  requires  that  the  examinee  use  natural  language  mode  to 
resolve  the  problems  of  the  simulated  patient.  No  direction  is 
provided,  such  as  multiple  choice  lists,  to  resolve  the  problem(s). 

COMPUTERS  AND  MEDICAL  INFORMATION  SCIENCE 

Osier  is  said  to  have  cited  ordered  and  logical  analysis  as  the  key 
to  good  patient  management  (68).  Today's  physicians  are  unable  to  con¬ 
sistently  perform  this  analysis  and  synthesis,  despite  their  improved 
training,  because  of  the  enormous  surge  of  knowledge  about  disease 
processes  and  therapeutics  in  the  past  30  years  and  the  inability  to 
bring  this  information  to  the  clinician  in  a  systematic  manner  at  the 
time  and  place  where  it  can  best  be  utilized.  Other  complicating 
factors  include  many  more  and  sophisticated  diagnostic  tests  and 
numerous  therapeutic  possibilities;  not  only  are  there  many  more  facts 
to  be  applied  to  patients'  problems,  but  many  more  facts  to  be  learned 
as  well.  The  end  result  has  been  a  conscious  effort  by  the  physician  to 
focus  his/her  sphere  of  responsibility  and  interest  through  special¬ 
ization,  such  that  only  a  portion  of  the  patient's  problems  can  be 
handled  by  any  one  individual. 

Even  though  computers  have  been  applied  to  various  aspects  of 
medicine,  they  have  only  recently  begun  to  influence  the  clinical 
practice  of  medicine  (25).  Patient  management  requires  numerous 
clinical  decisions  which  are  dependent  on  the  ability  to  systematically 
and  logically  process  increasing  amounts  of  information.  The  computer 
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can  influence  this  process  by  applying  sophisticated  and  widely  dis¬ 
persed  resources  more  effectively  and  by  assisting  in  the  basic 
decision-making  process. 

Recent  studies  support  the  idea  that  better  patient  management  can 
result  from  computer-assisted  strategy  applied  during  the  clinical 
patient  management  process.  McDonald  has  structured  a  feedback 
mechanism  in  the  form  of  computer-generated  recommendations  on  relative¬ 
ly  specific  decisions  regarding  individual  patient-management  situations 
(43).  Physicians  took  action  on  51  percent  of  the  recommendations  when 
these  were  communicated  in  the  form  of  cues,  and  on  only  22  percent  of 
the  equivalent  set  when  left  to  their  own  devices,  both  before  and  after 
receiving  feedback.  After  documenting  several  additional  studies  where 
various  clinical  cues  were  overlooked,  McDonald  concluded  that  limit¬ 
ations  in  physicians'  ability  to  react  consistantly  and  appropriately  to 
a  given  set  of  stimuli  in  the  clinical  setting  exist  by  virtue  of  being 
human,  a  fact  that  cannot  be  ignored. 

de  Dombal  has  demonstrated  that  the  accuracy  of  clinical  decisions 
supported  by  computer  logic  were  superior  to  the  clinicians'  working 
independently  on  600  cases  of  patients  complaining  of  acute  abdominal 
symptoms  (15,  16).  A  structured  information-acquisition  process  with 
feedback  to  the  clinicians  was  then  instituted  for  the  next  552  acute 
abdomen  cases  which  presented  at  the  St.  James  Hospital  in  Leeds, 

England.  Perforation  of  the  appendix  before  surgery  dropped  from  36 
percent  to  6  percent,  and  the  "negative”  laparotomy  rate  fell  from  20 
percent  to  7  percent.  When  structured  questions  and  feedback  were 

removed,  the  decision-making  of  the  clinicians  regressed  toward  previous 

» 

levels  (15,  16).  Subsequent  studies  by  the  same  group  in  1974  and  1975 


20 


corroborated  these  findings  (17). 

Computer-based  consultation  systems  range  from  those  used  in 
specific  disease  states  or  problem  areas  to  those  used  for  more  general 
purposes.  A  number  of  these  systems  are  summarized  below. 

Duke  University  Cardiovascular  Information  System 

This  system  provides  physicians  with  a  large  data  base  about 
clinical  experiences  with  coronary  artery  disease  (55).  The  data 
describe  outcomes  of  patients  with  various  sets  of  attributes.  The 
patient  attributes,  laboratory  and  physical  findings,  history  and 
outcome  of  a  large  number  of  patients  are  stored  in  a  computer,  which 
then  classifies  the  information.  When  attributes  of  a  new  patient  are 
entered,  the  computer  searches  all  patient  records,  and  those  which  fit 
the  criteria  are  then  separated  into  a  subgroup  for  further  on-line 
queries.  Rosati  and  his  group  have  likened  this  activity  to  a  "dynamic, 
living  computerized  medical  textbook".  It  has  the  advantage  of  giving 
the  physician  more  accurate  and  unbiased  information  based  on  a  large 
number  of  patients  (55). 

Electrolyte  and  Acid-Base  Consultation  System 

Bleich  has  constructed  a  computer-based  consultation  program  to 
help  physicians  manage  patients  with  electrolyte  and  acid-base  disorders 
(8).  The  program  directs  a  dialog  in  which  the  user  enters  clinical  and 
laboratory  information.  On  the  basis  of  the  abnormalities  detected,  the 
program  "asks"  questions  to  further  characterize  the  electrolyte  and 
acid-base  distrubance( s) .  During  or  after  the  completion  of  the  dialog, 
an  evaluative  note  is  produced  that  contains  a  list  of  diagnostic 
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possibilities,  an  explanation  of  underlying  pathophysiology,  therapeutic 
recommendations,  precautionary  measures,  suggestions  for  further 
studies,  and  references  to  the  medical  literature. 

HELP 

HELP  (Health  Evaluation  through  Logical  Processing)  is  a  complex 
diagnostic  management  support  system  at  the  Latter  Day  Saints  Hospital 
in  Salt  Lake  City,  Utah  (19).  A  variety  of  findings  on  symptoms  data 
and  patient  status  information  are  incorporated  into  the  system  (e.g. 
laboratory  tests,  blood  gas  analysis,  medical  record  abstracts,  etc). 
Decision  logic  utilizes  a  variety  of  statistical  techiques  to  determine 
probabilities  for  certain  diagnoses  or  treatment  selections  based  on  the 
historical  data  base.  In  addition,  the  system  provides  warning  of 
patient  conditions  that  may  require  intervention. 

A  survey  conducted  by  Batelle  Laboratories,  an  independent 
evaluation  group,  revealed  a  positive  attitude  among  physicians  toward 
the  HELP  system.  The  highest  proportion  of  physicians  that  were  very 
enthusiastic  about  the  system  were  specialists  in  surgery  and  internal 
medicine  (52). 

INTERNIST-I 

INTERNIST-I  was  designed  at  the  University  of  Pittsburg  as  a 
computer-based  diagnostic  system  for  problems  in  internal  medicine  (40). 
It  is  based  on  assigning  rough  estimates  of  the  likelihood  of  the 
association  of  a  disease,  given  a  particular  finding,  and  a  similar 
estimate  of  the  likelihood  of  a  finding,  given  a  particular  disease. 

The  current  knowledge  base  includes  roughly  70-75  percent  of  the  major 
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diagnoses  in  internal  medicine  (48).  INTERNIST-I  represents  an  attempt 
to  model,  within  a  computer  program,  the  thinking  processes  that  a 
clinician  uses  to  evaluate  a  case  and  make  a  diagnosis.  It  is  not  a 
model  based  on  probabilities  in  the  true  statistical  sense  and  there¬ 
fore,  is  subject  to  the  individual  perspective  brought  to  the  model. 

The  system  attempts  to  mimic  the  diagnostic  behavior  of  the 
"excellent  clinician".  Miller  et  al  attempted  to  document  the  strengths 
and  weaknesses  of  the  program  (48).  Its  performance  on  a  series  of  19 
clinicopathological  exercises  (Case  Records  of  the  Massachusetts  General 
Hospital)  published  in  the  New  England  Journal  of  Medicine  was 
qualitatively  similar  to  that  of  hospital  clinicans  but  inferior  to  that 
of  case  discussants.  The  evaluation  demonstrated  that  the  present  form 
of  the  program  is  not  sufficiently  reliable  for  clinical  applications. 
Specific  deficiences  that  must  be  overcome  include: 

1)  the  program's  inability  to  reason  automatically  or  temporally 

2)  its  inability  to  construct  differential  diagnoses  spanning 
multiple  problem  areas 

3)  its  occasional  attribution  of  findings  to  improper  causes,  and 

4)  its  inability  to  explain  its  "thinking". 

The  Indiana  University  Medical  Center  Computer  Reminders  System 

This  system  is  based  on  the  assumption  that  the  physician  must 
apply  a  few  simple  categorical  rules  to  a  few  items  of  information  many 
times  (43,  44).  Using  a  very  simple  computer  language,  designed 
specifically  for  this  system,  the  physician  writes  the  rules  he/she 
wishes  applied  to  the  data  and  then  lets  the  computer  repeatedly  apply 
them.  The  system  provides  reminders  for  a  large  percentage  of  simple 
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clinical  decisions;  it  assures  that  the  baseline  screening  tests  have 
been  done,  checks  that  abnormal  test  findings  are  examined  further,  and 
assumes  that  treatments  are  followed  with  appropriate  measures.  The 
system  occasionally  "suggests"  diagnoses  when  a  particular  abnormality 
is  evident,  but  more  often  suggests  treatments  that  might  correct  a 
pattern  of  abnormalities. 

PROM  IS 

The  Problem  Oriented  Medical  Information  System  at  the  University 
of  Vermont  was  developed  by  Weed  and  his  coworkers  to  facilitate  the 
organization  of  medical  care,  to  make  the  logic  of  health  care  decision 
explicit,  and  to  decrease  the  user's  reliance  on  memory  (14).  Although 
not  originally  designed  to  be  a  computer-based  consultation  system,  it 
is  capable  of  presenting  structured  reference  information  (3D. 

Data  is  organized  by  patient  problems.  The  computer  record  is 
structured  around  four  phases  of  medical  action: 

1)  an  initial  data  base  on  each  patient,  including  medical 
history  and  physical  examination 

2)  a  list  of  the  patient's  problems 

3)  diagnostic  and  treatment  plans  for  each  problem,  and 

4)  progress  notes  on  each  problem  indicating  how  the  patient  is 
progressing  during  therapy. 

Except  for  the  initial  data  base  every  entry  into  the  computer  record  is 
associated  with  a  particular  problem  of  the  patient.  By  structuring  the 
medical  record  in  this  way,  all  patient  information  is  logically 
organized  for  review. 

A  variety  of  diagnostic  and  management  protocols  based  on  common  or 
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potentially  serious  medical  problems  (e.g.  hypertension,  anemia, 
postoperative  fever,  and  cardiac  arrhythmias)  encountered  on  a  gyne¬ 
cology  service  have  also  been  developed.  This  information  is  available 
to  the  health  care  professional  via  a  touch  screen  cathode  terminal. 
Each  protocol  consists  of  hundreds  of  individual  displays.  The  content 
is  routinely  updated  by  a  committee  of  recognized  medical  authorities. 
The  updating  service  involves  comparing  the  information  contained  in 
articles  from  selected  medical  journals  with  the  computerized  medical 
displays  on  a  given  topic. 


MYCIN 

The  MYCIN  system  at  Stanford  University  is  an  interactive 
computer  program  which  uses  the  clinical  decision  criteria  of  experts  to 
advise  physicians  who  request  advice  regarding  selection  of  appropriate 
antimicrobial  therapy  for  hospital  patients  with  bacterial  infections 
(58).  The  system  uses  information  provided  by  the  physician  (e.g. 
signs,  symptoms,  laboratory  data,  and  normal  and  pathogenic  flora  of 
nonsterile  body  sites),  together  with  its  own  knowledge  base,  to 
determine  organism  significance,  organism  identification,  and 
appropriate  therapy  selection.  Through  interactive  prompting  the  user 
may  also  request  alternate  therapy,  recommendations,  explanations  of 
MYCIN’s  reasoning  processes  and  justifications  of  its  recommendations. 

Formal  evaluation  of  the  program's  therapy  recommendations  meet 
with  Stanford's  experts'  standards  of  accepted  practice  90%  of  the  time 
with  some  variation  noted  both  among  individual  experts  and  between 
Stanford  experts  and  others  (76). 


. 
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CARE 

The  CARE  (Clinical  Assessment,  Research  and  Educational)  system  is 
a  time-shared  computer-based  physiologic  monitoring  system  which  is 
available  nation-wide  in  the  United  States  (60).  It  is  designed  to  be  a 
management  and  education  aid  for  the  treatment  of  critically  ill 
surgical  patients. 

The  user  is  assisted  in  making  the  best  critical  care  decisions 
through  computer-directed,  interactive  prompting.  "Living  textbook" 
programs  present  therapeutic  cautions  and  recommendations  based  on  the 
initial  major  problem  and  underlying  factors.  References  to  relevant 
medical  literature  are  made  were  appropriate. 

In  a  nonrandomized ,  uncontrolled  prospective  analysis  of  all 
surgical  ICU  mortality,  Siegel  concluded  that  "the  application  of  the 
computer-based  CARE  system  for  physiologic  assessment  will  have  an 
additive  beneficial  effect  on  surgical  mortality,  when  used  in  the 
setting  of  a  structured  ICU  teaching  service  where  the  data  can  be 
appreciated  in  a  timely  fashion  and  used  as  important  information  by 
which  to  modify  therapeutic  decision  making"  (61).  Statistically 
significant  reductions  in  ICU  mortality  were  demonstrated  in  major 
traumatic  injury,  in  surgical  patients  with  complictions  of 
gastrointestinal  disease  and  major  noncardiac  thoracic  surgical 
procedures.  There  was  also  a  reduction  in  the  mean  surgical  ICU  length 
of  stay  for  survivors  of  0.82  days  and  therefore  of  estimated  cost  of 
care.  Whether  the  CARE  system  was  directly  or  indirectly  responsible 
for  these  findings  is  open  to  speculation.  Motivation  of  the  resident 
and  nursing  staff  and  organization  of  the  teaching  service  are  factors 
that  obviously  influence  the  success  of  the  CARE  system  but  attempts  to 
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evaluate  these  aspects  have  not  been  made. 

HOISS 

House  Officer  Information  and  Scheduling  System  (HOISS)  is  a  simple 
computer-based  problem  oriented  information  system  designed  to  assist 
physicians-in-training  with  the  investigation  and  treatment  of  acute 
medical  problems  (74).  It  consists  of  a  file  of  information  on  79 
common  medical  problems  ranging  from  raised  alkaline  phosphatase  to 
weight  loss.  The  information  on  each  problem  is  divided  into  general 
information,  investigative  procedures,  and  treatment. 

Using  information  provided  by  the  clinician,  HOISS  produces  an 
outline  of  each  of  the  patient’s  clinical  problems.  Hard  copies  of 
these  outlines  are  then  placed  in  the  patient's  record  and  are  used  as  a 
guide  to  collect  information  about  his/her  clinical  state. 

Evaluation  of  the  system  involved  tabulation  of  13  clinicians' 
responses  to  a  22-part  questionaire  (75).  All  the  users  found  the 
system  helpful  in  the  managment  of  patients'  problems,  felt  that  patient 
care  was  improved  and  that  their  knowledge  about  the  investigation  of 
acute  medical  problems  had  increased.  No  attempts  were  made  to 
objectively  determine  knowledge  acquisition  or  the  effect  of  HOISS  on 
patient  care. 

Young  also  compared  the  managment  of  similar  groups  of  patients 
cared  for  with  or  without  the  help  of  HOISS  (73).  He  concluded  that 
HOISS  increased  the  consistency  of  investigation  of  patients  with 
similar  problems.  Indeed,  HOISS  has  its  greatest  influence  on  less 


common  conditions. 
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THE  IMPACT  OF  COMPUTER  TECHNOLOGY  ON  CLINICAL  MEDICAL  INFORMATION 
SCIENCE 

"Computer  technology  and  information  science  have  been  applied  with 
recognized  success  to  many  areas  of  modern  scientific  and  industrial 
development  and  have  been  a  major  determinant  in  accelerated  pace  of 
advances  in  physical  and  basic  sciences.  However,  critical  applications 
in  the  area  of  medical  and  health  care,  although  frequently  supposed  and 
increasingly  attempted,  have  been  disappointing  in  their  impact  on  the 
health  care  system"  (29).  Freidman  and  Gustafson  (25)  have  outlined  the 
probable  reasons  for  the  failure  of  computer  technology  to  gain 
acceptance  among  physicians  and  to  influence  the  day-to-day  management 
of  patient  care: 

1)  The  physician  has  not  yet  been  provided  with  computer  systems 
that  exceed  his/her  own  capabilities.  In  medicine  the  measure 
of  success  is  diagnostic  accuracy  equal  to  that  of  a  skilled 
clinician,  accurate  EKG  analysis,  or  historical  data 
acquisition  which  saves  the  physician  time.  Many  physicians 
feel  that  computers  are  professionally  threatening  to  their 
role  as  clinicians  and  will  only  serve  to  further  dehumanize 
the  health  care  system. 

2)  The  physician-computer  interaction  is  unnatural  and  can  be 
tedious  and  time-consuming.  Computer  systems  require 
physicians  and  other  health  care  professionals  to  work  in  a 
manner  which  is  alien  to  their  normal  practice  and/or  training. 

3)  A  significant  positive  impact  on  patient  care  by  successful 
utilization  of  computer  technology  on  medicine  and  health  care 
has  yet  to  be  demonstrated.  The  initial  construction  and 
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implementation  of  a  computer  system  is  expensive, 
time-consuming  and  may  require  reorganization  and/or  retraining 
of  hospital  personnel. 

4)  There  is  a  disarray  of  computer  languages  and  computer  systems 
that  prevents  the  transfer  of  applications  from  one  institution 
to  another.  Unlike  business  and  industry,  most  computer 
systems  are  not  compatible  or  interchangeable. 

5)  Research  has  not  been  conducted  in  satisfactory  manner.  The 
majority  of  projects  in  the  area  of  computer  applications  to 
medicine  subsequently  prove  to  be  impractical,  too  expensive, 
or  unacceptable.  And  yet  researchers  continue  to  duplicate 
previous  efforts  or  mistakes.  A  survey  of  32  computer 
applications  in  medicine  published  during  a  5-year  period 
revealed  that  51  percent  of  these  projects  had  been  abandoned 
or  temporarily  halted  when  the  initial  research  funds  were 
exhausted  (25).  This  occurred  despite  the  fact  that  the 
majority  (63  percent)  of  the  projects  had  lived  up  to  their 
author's  expectations.  In  almost  every  case  the  reason  for 
abandonment  was  that  the  project  was  not  cost-effective.  In 
only  19  percent  of  cases  did  the  authors  of  the  programs  report 
that  they  were  in  routine  use. 

These  reasons  are  at  least  part  of  the  explanation  for  the  lack  of 
widespread  acceptance  of  clinical  computer  systems.  Little  information 
is  available  on  the  evaluation  of  such  systems.  This  thesis  describes 
the  development  and  evaluation  of  a  computer-based  bedside  information 
system  in  the  intensive  care  unit.  The  system  is  called  the  CLINICAL 


REFERENCE  LIBRARY. 
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SUMMARY 

Literature  related  to  the  role  of  computers  in  medicine  was 
presented  in  this  chapter.  Major  clinical  information  systems  available 
today  were  discussed.  The  lack  of  ICU-directed  or  -oriented  clinical 
information  systems  was  emphasized.  The  reasons  for  the  failure  of 
computer  technology  to  gain  acceptance  within  the  medical  community  were 
also  summarized.  The  next  chapter  describes  the  environment  in  which 
the  CLINICAL  REFERENCE  LIBRARY  was  tested. 


CHAPTER  III 


THE  INTENSIVE  CARE  DELIVERY  SYSTEM 
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This  chapter  is  divided  into  three  parts.  The  development  of  the 
specialty  of  critical  care  medicine  and  the  concept  of  the  intensive 
care  unit  are  described  in  part  one.  Part  two  is  devoted  to  the 
description  of  the  intensive  care  delivery  system  at  the  University  of 
Alberta.  The  organization  of  the  intensive  care  unit  and  the  structure 
of  the  teaching  services  are  also  described.  The  chapter  is  summarized 
in  part  three. 

INTRODUCTION 

The  techniques  available  today  for  preserving  vital  functions  in 
the  critically  ill  or  injured  patient  were  largely  unknown  before  1950 
when  a  conservative  approach  characterized  the  management  of  this  group 
of  patients.  It  was  not  until  the  1960's  that  intubation  and  mechanical 
ventilation,  hemodialysis,  volume  repletion  guided  by  central  venous 
pressure  and  cardiac  resuscitation  by  the  routine  use  of  defibrillators 
and  pacemakers  came  into  general  use.  Progress  in  the  field  has  been 
accelerated  by  giant  strides  in  technology. 

The  intensive  care  unit  (ICU)  provides  personnel  and  facilities 
that  make  it  possible  to  bring  the  full  benefits  of  these  and  other 
techniques  to  the  critically  ill  or  injured  patient  (70).  The  physician 
who  is  called  to  the  bedside  is  confronted  by  an  array  of  monitors  and 
sophisticated  equipment.  Although  the  traditional  techniques  of 
noninvasive  diagnosis  are  essential  for  competent  understanding  of  the 
patient’s  clinical  status  they  do  not  suffice.  In  times  of  crisis, 
however,  specific  measurements  of  major  physiologic  distrubances  and 
quantitation  of  the  severity  of  the  vital  defects  are  needed. 

Practical  examples  may  be  cited  to  illustrate  the  implications  of 
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expanded  concepts  of  diagnosis  and  monitoring.  Auscultatory  changes  in 
the  chest,  peripheral  cyanosis  and  other  physical  findings  are  likely  to 
be  supplementary  rather  than  primary  indicators  of  the  patient's 
physiological  state.  The  physician  or  nurse  must  be  prepared  to  obtain 
arterial  blood  samples  for  analysis  of  blood  gases  and  blood  pH. 
Alterations  in  the  cardiac  rhythm  can  be  detected  by  cardiac  ausculta¬ 
tion,  but  they  are  interpreted  most  appropriately  on  the  basis  of  the 
electrocardiogram,  especially  in  relation  to  the  central  arterial 
pulses.  Neck  vein  distention  and  hepatic  enlargement  are  not  sensitive 
signs  of  limited  cardiac  competence  in  the  critically  ill  patient;  the 
clinician  may  be  called  on  to  insert  a  catheter  into  the  right  heart, 
and  into  the  pulmonary  artery  to  obtain  pertinent  information  on 
ventricular  function  which  is  then  used  to  determine  clinical 
management.  Guided  by  a  combination  of  clinical,  physiologic  and 
laboratory  findings,  the  physician  working  in  the  intensive  care  unit  is 
prepared  to  insert  an  endoctracheal  tube  and  to  institute  mechanical 
ventilation.  In  instances  of  acute  circulatory  failure,  he  increases 
the  intravascular  volume  by  "fluid  challenging",  at  the  same  time, 
guarding  against  the  risk  of  pulmonary  edema  and  potentially  fatal  acute 
respiratory  failure.  He  supports  myocardial  function  by  the 
administration  of  inotropic  agents  such  as  dopamine  or  by  mechanical 
assist  devices  such  as  the  intraaortic  balloon  pump.  If  a 
life-threatening  alteration  in  cardiac  rhythm  presents  itself,  he  must 
be  prepared  to  use  either  pharmacologic  or  electric  methods  by  which  an 
effective  rhythm  is  restored.  In  summary,  his  competence  crosses 
traditional  specialty  lines. 
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THE  UNIVERSITY  OF  ALBERTA  HOSPITALS  INTENSIVE  CARE  DELIVERY  SYSTEM 

Organization 

The  Univerity  Hospital  is  the  primary  teaching  hospital  of  the 
Faculty  of  Medicine  at  the  University  of  Alberta.  It  serves  as  the 
major  referral  institution  for  Northern  Alberta.  The  Royal  Alexandra 
Hospital  is  a  center  for  secondary  and  tertiary  care  and  is  also 
affiliated  with  the  Faculty  of  Medicine. 

The  University  Hospital  ICU  (Station  68)  is  a  10-bed  unit  capable 
of  providing  full  vital  systems  monitoring  and  organ  support.  It  is  not 
a  postoperative  recovery  room  or  a  coronary  care  unit.  It  operates  as  a 
"closed  unit"  in  that  all  patients  are  transferred  to  the  care  of  the 
Director  or  one  of  the  Associate  Directors  and  are  cared  for  by  the  ICU 
team.  This  multidisciplinary  team  consists  of  a  Chief  Resident,  three 
or  four  Rotating  Residents  (secunded  from  Internal  Medicine,  Anesthesia, 
and  General  Surgery  and  the  other  surgical  subspecialties) ,  a  Charge 
Nurse,  Nurse  Specialists  in  Critical  Care,  Respiratory  Technologists, 
Chest  Physiotherapists  and  a  Life  Support  Technician. 

The  ICU  at  the  Royal  Alexandra  Hospital  (Station  23)  is  an  8-bed 
general  systems  failure  unit  under  the  direction  of  a  Director  and  a 
Co-director.  It  is  neither  a  coronary  care  unit  nor  a  general  recovery 
room  although  postoperative  craniotomy  cases  are  routinely  admitted.  It 
is  also  a  "closed  unit".  The  multidisciplinary  team  is  identical  to 
that  of  the  University  Hospital  ICU  except  that  there  is  no  Life  Support 
Technician . 

The  ICU  team  is  responsible  for  all  aspects  of  care  of  the 
critically  ill  or  injured  patient.  Both  units  have  facilities  for 
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monitoring  and  support,  including  mechanical  ventilation,  flow-directed 
balloon-tipped  catheter  monitoring,  fiberoptic  bronchoscopy,  intra¬ 
cranial  pressure  monitoring,  total  parenteral  and  enteral  nutrition, 
peritoneal  dialysis,  and  cardiac  pacing.  The  University  Hospital  ICU 
has  facilities  for  hemodialysis,  hemoperf usion ,  plasmaphoresis ,  and 
external  counterpulsation. 

There  are  two  major  differences  in  the  organization  of  the 
University  Hospital  ICU  and  the  Royal  Alexandra  Hospital  ICU:  the  former 
conducts  an  air  ambulance  service  for  Northern  Alberta  while  the  latter 
does  not  routinely  care  for  patients  under  the  age  of  sixteen  years. 

The  ICU  Teaching  Service 

The  intensive  care  unit  plays  an  important  role  in  undergraduate 
and  postgraduate  (residency)  training.  Three  or  four  second-  or 
third-year  residents  are  assigned  to  the  intensive  care  unit  for  2-  to 
3-month  overlapping  rotations.  These  residents  have  no  other  patient 
care  responsibilities.  All  residents  are  assigned  to  the  unit  during 
the  day  and  one  resident  remains  on-call  in  the  unit  each  night. 

Although  the  overall  responsibility  for  and  direction  of  patient  care  is 
that  of  the  Director  or  one  of  the  Associate  Directors  (or  Co-director) , 
the  details  of  patient  management  are  the  responsibility  of  the  Chief 
Resident  and  the  ICU  Residents.  All  orders,  with  appropriate 
supervision,  are  written  by  the  ICU  housestaff.  The  ultimate  authority 
and  responsibility  for  changes  in  therapy  as  recommended  by  consultants 
not  associated  with  the  ICU  is  that  of  the  Director  or  one  of  his 
associates . 


Decisions  reserved  for  the  Chief  Resident  or  the  Directors  are 
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concerned  with  the  appropriate  utilization  of  ICU  resources,  including 
triage  and  the  institution  and  the  countermanding  of  therapeutic 
decisions  which  are  contrary  to  general  ICU  policy  or  are  detrimental  to 
patient  care. 

Medical  students  are  usually  relegated  to  the  role  of  observers  but 
are  encouraged  to  participate  in  the  daily  routine  of  the  intensive  care 
unit  as  much  as  possible. 

Bedside  rounds  are  conducted  at  least  twice  daily.  Weekly  service 
rounds  are  conducted  by  the  resident  staff  and  medical  students. 

Didactic  sessions  are  held  daily  and  are  presented  by  the  residents, 
attending  staff  or  visiting  professors. 

Nursing  inservice  rounds  are  held  at  each  hospital  on  a  regular 
basis.  Even  though  attendance  is  mandatory  at  the  Royal  Alexandra 
Hospital,  attendance  at  such  sessions  are  about  equal. 

The  total  number  of  patients  admitted  to  each  ICU  each  year  is  in 
the  order  of  450.  The  mean  ICU  stay  is  about  7  days  with  a  median  of 
3.5  days.  Patients  come  for  medical  and  surgical  services  in 
approximately  equal  numbers. 

SUMMARY 

In  this  chapter  the  intensive  care  delivery  system  at  the 
University  of  Alberta  was  described.  The  next  chapter  deals  with  the 
HP  5600A  Computerized  Patient  Data  Management  System. 


CHAPTER  IV 

THE  HP  5600A  COMPUTERIZED  PATIENT  DATA 
MANAGEMENT  SYSTEM 
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In  this  chapter  four  topics  are  discussed.  The  first  part  is  a 
brief  overview  of  the  capabilities  of  the  HP  5600A  Patient  Data  Manage¬ 
ment  System:  the  patient  monitoring  system  with  which  the  CLINICAL 
REFERENCE  LIBRARY  is  intimately  associated.  The  second  part  describes 
the  system  hardware.  Operation  of  the  CLINICAL  REFERENCE  LIBRARY  from 
the  bedside  is  emphasized.  A  brief  description  of  the  software  in¬ 
cluding  the  CLINICAL  REFERENCE  LIBRARY  is  presented  in  the  third  part. 
The  chapter  is  summarized  in  part  four. 

INTRODUCTION 

The  HP  5600A  Patient  Data  Management  System  (PDMS)  was  installed  in 
the  University  Hospital  ICU  in  November  1980.  The  initial  purchase 
price  was  $425,000.  Operating  and  maintenance  costs  are  about  $30,000  a 
year.  Operation  of  the  system  is  the  responsibility  of  the  Department 
of  Information  Systems. 

THe  PDMS  handles  a  wide  spectrum  of  patient  data  from  admission 
through  diagnosis  and  treatment,  to  discharge  and  followup.  It 
automatically  logs  each  patient’s  vital  signs,  accepts  and  stores  staff 
notes  and  manually-entered  laboratory  data,  produces  a  variety  of 
displays  of  data  in  tabular  and  trend  form,  and  prints  hard  copies  of 
the  displays  as  well  as  other  reports  on  demand. 


THE  HARDWARE 

Conceptually,  the  PDMS  consists  of  a  medical  subsystem  and  a 
computer  subsystem.  The  medical  subsystem  consists  of  bedside 
monitoring  equipment,  medical  data  terminals  and  the  hardware  necessary 
to  interface  the  monitors  and  terminals  to  the  computer.  The  211 3 A 
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computer  and  the  equipment  peripheral  to  it  comprise  the  computer 
subsystem  (Plate  1).  The  computer  utilizes  a  HP  7920  disc  drive 
(capacity  50  Mbytes)  and  drives  a  HP  2631A  character  printer. 

The  beside  monitors  have  direct  cabling  to  the  central  nursing 
station  and  to  the  computer’s  analog-digital  converter.  One  terminal 
can  be  shared  by  several  beds.  Alpha-numeric  displays  with  attached 
keyboards  are  located  in  the  computer  room  (Plate  1)  and  central  nursing 
station  (Plate  2)  and  can  be  connected  to  the  computer  via  telephone. 

All  communication  between  the  user  and  the  PDMS  is  conducted  in 
natural  language  mode.  There  are  two  basic  types  of  terminals:  one,  the 
HP  5671C  Video  Monitor  is  located  at  the  bedside  and  consists  of  a  hand 
held  keyboard  (Plate  3)  and  a  14"  video  screen  (Plate  4);  the  other,  the 
alpha-numeric  terminal  (models  HP  2645A  and  HP  2648A)  is  found  in  the 
central  nursing  station  and  computer  room  and  consists  of  a  5  x  10-inch 
video  display  in  a  keyboard  similar  to  that  of  a  standard  typewriter 
(Plate  5).  Certain  keys  on  both  keyboards  are  designated  as  special 
function  keys  and  are  identified  by  a  function  name  engraved  in  color 
above  the  key's  standard  character.  The  purpose  of  such  keys  is  to 
provide  quick,  direct  access  to  specific  data  displays. 

SYSTEM  SOFTWARE 

Software  consists  of  system  programs  written  in  Assembler  language 
and  application  programs  written  in  FORTRAN. 

All  data  generated  during  hospitalization  in  the  ICU  can  be 
recorded.  These  include  patient  admission  information,  clinical 
history,  physical  examination,  routinely  monitored  signals  (heart  rate, 
systolic,  diastolic,  mean  arterial  and  central  venous  pressures,  EKG, 
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Plate  1: 

The  2113A  HP  Computer  and 
Peripheral  Hardware 
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Plate  2: 

The  Central  Nursing  Station 
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Plate  3: 

The  HP  5671C  Video  Monitor: 
Handheld  Keyboard 


Plate  4: 

The  HP  567 1C  Video  Monitor 
Video  Screen 
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Plate  5: 

The  HP  2645A  Alpha-Numeric  Terminal 
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and  core  temperature),  physicians'  notes  and  nurses'  progress  notes, 
and  manually-entered  data,  including  laboatory  results,  X-ray  reports, 
fluid  balance  and  respiratory  information.  The  complete  record  of  each 
patient  is  available  for  on-line  retrieval  and  hard  copies  of  displays 
or  printed  reports  are  easily  obtained. 

The  ICU  team  exercises  complete  control  over  all  aspects  of  patient 
monitoring,  from  selection  of  the  physiological  parameters  to  be 
monitored  and/or  trended,  to  suspension  or  resumption  of  monitoring. 

All  control  is  done  dynamically,  without  stopping  or  interrupting  the 
system.  Data  may  be  stored  temporarily  or  permenantly. 

Useful  hemodynamic  variables  (cardiac  index,  stroke  volume  index, 
systemic  vascular  resistance  index,  pulmonary  vascular  resistance  index) 
can  be  derived  from  monitored  vital  signs  data  and  manually-entered  data 
pertinent  to  a  specific  cardiac  output  as  determined  by  the  thermal 
dilution  technique. 

The  Patient  Management  subsystem  enables  the  user  to  flip  rapidly 
back  and  forth  through  data  pertinent  to  a  particular  patient's  problem, 
much  as  one  would  flip  through  a  patient's  chart.  The  problems  relevant 
to  the  patient  are  selected  from  a  master  list  of  problems. 

The  Drug  Infusion  Calculator  subsystem  provides  a  useful  and 
time-saving  means  of  rapidly  calculating  the  dose  or  rate  of  drug 
infusion.  Features  of  this  subsystem  include  a  list  of  commonly  used 
drugs,  a  free-form  text  of  recommendations  (i.e.  information  on  admin¬ 
istration  and  drug  action) ,  a  dose  meter  which  graphically  illustrates 
the  dose  to  be  administered  and  warning  messages  which  are  displayed  to 
alert  the  user  to  toxic  doses,  erroneous  entries  or  missing  information. 

The  HP  Reference  Library  provides  a  flexible  means  for  storing 
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information  which  may  later  be  recalled  for  the  purpose  of  references, 
training  or  review. 

The  CLINICAL  REFERENCE  LIBRARY  is  available  at  all  terminals 
including  the  bedside  terminals. 

SUMMARY 

In  this  chapter  the  HP  5600A  Computerized  Patient  Data  Management 
System  of  which  the  Clinical  Reference  Library  is  intimately  associated 
was  described.  The  next  chapter  describes  the  development  of  the 
CLINICAL  REFERENCE  LIBRARY. 


CHAPTER  V 

THE  CLINICAL  REFERENCE  LIBRARY 
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This  chapter  which  describes  the  development  of  the  CLINICAL 
REFERENCE  LIBRARY  is  divided  into  five  parts.  Part  one  describes  the 
conceptual  framework  and  philosophy  of  the  CLINICAL  REFERENCE  LIBRARY. 
Part  two  is  devoted  to  the  engineering  considerations.  Development  of 
the  clinical  information  "modules"  is  presented  in  part  three.  Part 
four  describes  the  operation  of  the  CLINICAL  REFERENCE  LIBRARY.  The 
chapter  is  summarized  in  part  five. 

INTRODUCTION 

As  was  mentioned  earlier,  one  of  the  problems  confronting 
physicians  today  is  that  of  keeping  abreast  of  medical  information. 
Bernier  has  stated  that  the  amount  of  information  to  be  processed  and 
assimilated  is  well  beyond  our  abilities  and  that  our  current  retrieval 
processes  are  too  cumbersome  (6).  He  has  quoted  Goudsmit:  "A  modern 
computer  can  make  a  complete  literature  search  in  a  remarkable  short 
time  and  overwhelm  one  with  a  bibliography  that  requires  a  lifetime  to 
read  and  study." 

Bernier  has  stated  that  what  is  needed  is  a  "genuine  information 
retrieval  system"  (6).  That  is,  one  which  "answers"  questions  immed¬ 
iately.  He  has  suggested  that  the  following  features  be  incorporated 
into  such  an  information  retrieval  system: 

1)  the  material  presented  should  be  indexed 

2)  the  subject  matter  should  be  limited  to  specific  areas 
designated  by  the  user  group  and  should  be  updated  regularly 

3)  the  subject  matter  should  be  an  abstract  of  the  original 
work( s) 

4)  the  mechanism  by  which  the  information  is  presented  should  be 
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physically  accessible  at  all  times,  and 

5)  the  information  must  be  available  at  all  times. 

These  charater istics  were  incorporated  into  the  CLINICAL  REFERENCE 
LIBRARY. 

ENGINEERING  CONSIDERATIONS 

The  philosophy  presiding  over  the  design  of  the  CLINICAL  REFERENCE 
LIBRARY  was  to  minimize  the  technical  burden  associated  with  the 
development,  maintenance,  and  use  of  the  computerized  medical  informa¬ 
tion  "modules". 

The  use  of  nonskilled  users  requires  that  the  CLINICAL  REFERENCE 
LIBRARY  be  menu-driven.  Explicit  instructions  are  available,  at  every 
step  in  the  CLINICAL  REFERENCE  LIBRARY.  This,  combined  with 
user-oriented  displays,  provides  the  user  with  the  information  needed  to 
pinpoint  his/her  position  in  the  CLINICAL  REFERENCE  LIBRARY  at  all 
times.  The  ability  to  exit  a  module  from  any  level  at  any  time  and  to 
access  any  module  are  also  important  in  this  respect. 

All  the  information  in  the  CLINICAL  REFERENCE  LIBRARY  file  is 
stored  on  magnetic  tape  and  is  available  24  hours  a  day.  The  response 
time  of  the  user-computer  interactions  is  very  rapid.  The  system  is 
self-instructing . 

Because  of  the  highly  illustrative  nature  of  much  of  what  is 
presented  in  traditional  medical  and  allied  health  education,  graphics 
are  a  necessary  component  of  the  CLINICAL  REFERENCE  LIBRARY. 
Unfortunately,  only  one  graphics  terminal  (model  HP  2648A)  has  been 
installed  in  the  University  Hospital  ICU.  All  graphics  are  therefore 
presented  to  the  user  on  paper.  A  107-page  binder  containing  75 
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laminated  illustrations  and  1 1  tables  designed  by  the  author  are 
immediately  adjacent  to  all  computer  terminals. 

DEVELOPMENT  OF  INFORMATION  MODULES 

The  content  of  the  clinical  information  modules  was  based  on 
consideration  of  common  and  potentially  serious  medical  problems 
frequently  encountered  in  the  intensive  care  unit.  The  charts  of  75 
patients  admitted  to  the  University  Hospital  ICU  during  the  first  2 
months  of  1981  were  reviewed.  Twenty  disease  processes  or  clinical 
problems  were  chosen  and  developed  for  the  CLINICAL  REFERENCE  LIBRARY 
from  this  chart  review  (Table  1). 

Each  module  consists  of  many  computer  displays  (ranging  from  5  to 
186)  and  contains  information  needed  for  optimal  patient  care.  Where 
applicable,  information  basic  for  any  single  problem  was  classified 
under  the  following  headings: 

1)  definition 

2)  incidence 

3)  etiology 

4)  bacteriology/ pathology/ pathophysiology 

5)  clinical  course 

6)  diagnosis 

7)  treatment 

8)  complications 

9)  prognosis 

10)  references 
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Table  1 

Content  of  the  CLINICAL  REFERENCE  LIBRARY 

1.  The  problem  oriented  medical  record 

2.  Airway  management  including  intubation  and  tracheostomy 

3.  Pulmonary  physiology 

4.  Acid-base  regulation 

5.  Gas  transport 

6.  The  pathophysiology  of  acid-base  abnormalities 

7.  Pulmonary  embolism 

8.  Aspiration  pneumonia 

9.  Septic  shock 

10.  Adult  Respiratory  Distress  Syndrome  (ARDS.) 

11.  Pediatric  medications  including  dosages,  intervals  and  routes 
of  administration 

12.  Drug  interactions  including  adverse  effects  and  possible 
mechani sms 

13.  Acute  poisonings  (acetaminophen,  barbiturates,  acid  alcohols, 
carbon  monoxide,  salicylates  and  tricyclic  antidepressants) 

14.  Antimicrobial  therapy 

15.  The  pathophysiology  of  starvation  and  the  principles  of 
hyperalimentation 

16.  Brain  resuscitation  and  ICP  monitoring 

17.  Flow-directed  balloon-tipped  catheter  monitoring 

18.  Hypothermia 

19.  Burns 

20.  The  pathophysiology,  diagnosis  and  management  of  cardiac 


arrhythmias 
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The  "common  format"  is  based  on  McWhinney ' s  analysis  of  common 
problem-solving  errors  (45): 

1)  the  failure  to  respond  to  cues  presented  by  the  patient  due  to 
physician  inexperience  or  due  to  the  mental  set  of  the 
physician 

2)  premature  convergence  on  a  diagnostic  hypothesis  and  failure 
to  consider  alternate  hypotheses 

3)  errors  in  diagnostic  strategy  (i.e.  redundancy  and  inadequacy 
of  testing) ,  and 

4)  management  errors. 

It  is  apparent  from  this  list  that  failure  to  remain  well  informed  can 
be  related  directly  to  these  problem-solving  errors.  Conversely,  these 
errors  are  related  to  the  specific  components  of  the  "common  format" 
data  base. 

The  modules  were  developed  during  the  latter  6  months  of  1981  and, 
except  for  occasional  revisions  based  on  user  suggestions  (e.g.  spell¬ 
ing  and  punctuation)  and  updating  of  the  references,  remained  stable 
during  the  period  of  the  study.  Creation  and  updating  of  the  modules 
were  the  responsibility  of  the  author.  Information  from  recent 
textbooks  and  journal  articles  was  converted  into  workable  computer 
displays  with  appropriate  reference  citations.  An  audit  of  the  medical 
content  was  completed  before  the  modules  were  incorporated  into  the 
CLINICAL  REFERENCE  LIBRARY.  Recognized  and  respected  authorities  in 
various  medical  and  surgical  specialties  were  invited  to  criticize  the 
content  of  the  modules  for  accuracy,  currency  and  completeness.  The 
author  reviewed  their  suggestions  and  criticisms.  Appropriate  changes 
were  made  where  necessary.  Opposing  views  and  opinions  were  expressed 
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in  areas  of  controversy  (i.e.  the  best  possible  therapy,  recommended 
practices  ,  etc  . ) . 

Revision  of  the  text  can  be  easily  acommplished  with  the  use  of  a 
visual  editor.  This  does  not  require  disruption  of  the  CLINICAL 
REFERENCE  LIBRARY.  Changes  in  the  programming  of  the  prompts,  etc.  can 
be  easily  performed  using  preconstructed  utility  programs. 


SYSTEM  OPERATION 

The  author  assumes  that  the  user  is  totally  naive  about  computers. 
Consequently,  equipment  is  limited  to  three  familar  PDMS  elements: 

1)  a  television  monitor 

2)  a  hand-held  keyboard  reminiscent  of  the  touch-tone  telephone, 
and 

3)  a  typewriter  keyboard  (of  which  only  14  keys  are  necessary  for 
operation  of  the  CLINICAL  REFERENCE  LIBRARY). 

The  level  of  understanding  need  be  no  greater  than  the  ability  to 
read  and  follow  directions.  Operation  of  the  CLINICAL  REFERENCE 
LIBRARY  requires  only  six  simple  commands  (APPENDIX  1). 

When  a  resident  decides  that  the  clinical  management  of  a 
critically  ill  patient  is  likely  to  be  difficult  or  that  he/she  lacks 
sufficient  knowledge  about  a  particular  clinical  problem,  he/she  can 
consult  the  CLINICAL  REFERENCE  LIBRARY.  After  entering  his/her  personal 
identification  number  the  user  is  immediately  presented  with  an  index  or 
"main  menu"  of  twenty  problems  or  problem  areas  ("modules")  (APPENDIX 
2).  Having  typed  a  module  number,  for  example  17,  he/she  is  presented 
with  an  index  to  "ACUTE  POISONINGS"  (APPENDIX  3).  The  user  then  has 


several  options: 
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1)  he/she  can  start  at  the  beginning  of  the  module  and  continue 
through  the  module  as  if  he/she  were  reading  a  book,  or 

2)  he/she  can  directly  access  a  specific  heading  or  subheading 
e.g.  17.1.5  TREATMENT  (OF  ACETAMINOPHEN  POISONING)  and  continue 
through  that  section  as  if  he/she  were  reading  a  chapter  in  a 
book,  or 

3)  he/she  can  directly  access  a  specific  page  within  the  module 
and  continue  on  from  there  e.g.  on  page  9,  entitled  17.1.3 
PATHOLOGY  AND  PATHOPHYSIOLOGY  (OF  ACETAMINOPHEN  POISONING),  the 
major  pathophysiological  aspects  are  presented. 

After  using  the  CLINICAL  REFERENCE  LIBRARY,  the  user  can 
immediately  sign-off  or,  more  preferably,  can  access  a  nine-part 
questionaire  (APPENDIX  4). 


SUMMARY 

This  chapter  was  devoted  to  the  description  of  the  CLINICAL 
REFERENCE  LIBRARY.  The  conceptual  framework  and  philosophy,  engineering 
considerations  and  development  of  the  clinical  information  modules  were 
described.  Operation  of  the  CLINICAL  REFERENCE  LIBRARY  was  also 
presented.  The  next  chapter  describes  the  methodology  employed  to 
evaluate  the  effectiveness  of  the  CLINICAL  REFERENCE  LIBRARY  as  a 


teaching  tool. 


CHAPTER  VI 

MATERIALS  AND  METHODS 
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In  this  chapter  the  methodology  employed  to  evaluate  the  effective¬ 
ness  of  the  CLINICAL  REFERENCE  LIBRARY  as  a  teaching  tool  is  described. 
The  chapter  is  divided  into  five  parts.  Part  one  describes  the  experi¬ 
mental  design.  Part  two  describes  the  participants.  Qualitative  and 
quantitative  data  gathering  techniques  are  described  in  part  three. 

Part  four  documents  how  the  results  of  the  pre-  and  post-test 
examinations  and  questionaires  are  combined  in  the  final  analysis.  The 
chapter  is  summarized  in  part  five. 

DESCRIPTION  OF  PARTICIPANTS 

Sixty-nine  health  care  personnel  were  evaluated  prospectively  over  a 
12-month  period.  Forty-six  participants  were  assigned  to  the  control 
group  which  was  composed  of  8  medical  students,  9  nurses  and  10 
residents  from  the  Royal  Alexandra  Hospital  ICU  and  8  medical  students, 

4  nurses  and  7  residents  from  the  University  Hospital  ICU.  Twenty-three 
health  care  personnel  were  assigned  to  the  experimental  group  which  was 
composed  of  8  medical  students,  8  nurses  and  7  residents  from  the 
University  Hospital  ICU. 

Three  or  four  second-  or  third  year  residents  were  assigned  to  each 
ICU  for  2-  or  3-month  overlapping  rotations.  The  resident  population 
was  composed  of  trainees  in  Internal  Medicine,  General  Surgery,  Ortho¬ 
pedic  Surgery  and  Anesthesia.  Prior  experience  in  critical  care 
medicine  or  allied  fields  was  variable.  Most  residents  had  had  exper¬ 
ience  in  at  least  one  of  the  following:  coronary  care,  neonatology, 
post-operative  cardiac  recovery,  anesthesia  or  adult  critical  care. 

One  or  two  third  or  fourth  year  medical  students  were  assigned  to 
either  ICU  for  four  weeks  at  a  time.  Prior  experience  in  critical  care 
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medicine  was  negligible  in  this  subgroup. 

Only  nurses  permenantly  working  in  the  intensive  care  unit  were 
involved  in  this  study.  Nurses  not  trained  in  critical  care  were  not 
eligible.  All  the  nurses  had  similar  backgrounds  and  work  experience. 

EXPERIMENTAL  DESIGN 

The  study  was  a  classical,  non-randomized ,  pre-test-post-test, 
control-experiment  research  paradigm.  This  design  is  one  in  which  a 
group  which  has  experienced  "X"  (in  this  case,  the  CLINICAL  REFERENCE 
LIBRARY)  is  compared  with  one  which  has  not,  for  the  purpose  of  estab¬ 
lishing  the  effect  of  "X"  (11).  This  is  graphically  illustrated  as: 

0X0 
0  0 

In  the  interest  of  the  time,  the  study  was  divided  into  two  phases. 
During  the  first  phase,  all  participants  entering  the  study  were 
assigned  to  the  control  group  (August  16,  1981  -  February  14,  1982). 
This  part  of  the  study  was  conducted  during  the  development  of  the 
CLINICAL  REFERENCE  LIBRARY  and  was  terminated  as  soon  as  it  became 
operational.  During  the  second  phase  (February  15,  1982  -  August  15, 
1982),  all  participants  entering  the  study  were  assigned  to  the 
experimental  group  and  were  allowed  unlimited  access  to  the  CLINICAL 
REFERENCE  LIBRARY.  Participants  were  not  randomly  assigned  to  either 
ICU  or  either  control  or  experimental  group.  Resident  and  medical 
student  assignment  was  determined  by  the  Director  of  Resident  Training 
and  the  Phase  III  Co-ordinator,  respectively.  Neither  official  was 
affiliated  with  the  study. 

All  participants  were  pre-  and  post-tested  using  the  same  multiple 
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choice  question  examination.  Medical  students  and  residents  were  tested 
no  later  than  72  hours  after  commencing  their  ICU  rotation  and  no  later 
than  24  hours  after  leaving  the  service.  The  average  interval  between 
pre-  and  post-testing  was  4  weeks  and  10  weeks  for  medical  students  and 
residents,  respectively.  The  third  subgroup,  the  nurses,  was  somewhat 
different  in  that  at  the  time  of  testing  many  had  already  been  working 
in  the  ICU  for  several  years.  Because  the  nurses  also  worked  shift 
work,  they  were  pre-  and  post-tested  at  their  convenience.  The  average 
interval  between  tests  for  the  nurses  was  12  weeks. 

During  the  course  of  the  study  both  groups  were  encouraged  to  use 
conventional  methods  of  learning  e.g.  textbooks,  journals,  bedside 
rounds,  and  consultation  with  staffmen  and  fellow  colleagues.  Thus,  the 
only  substantial  difference  between  the  two  study  groups  involved 
exposure  to  the  CLINICAL  REFERENCE  LIBRARY. 

INSTRUMENTS 

The  Multiple  Choice  Question  Examination 

A  comprehensive  multiple  choice  question  examination  was  designed  to 
examine  knowledge  in  the  field  of  critical  care  medicine  and  to  measure 
the  acquisition  of  such  knowledge  after  working  in  the  ICU  for  a 
predetermined  time.  The  questions  were  designed  to  not  only 
discriminate  between  the  major  groups  of  participants  but  also  between 
the  various  subgroups  (e.g.  medical  residents  versus  surgery  residents 
and  senior  nurses  versus  nurses  with  minimal  experience) . 

The  examination  covered  pulmonary  physiology,  emergency  resuscita¬ 
tion,  infection  and  bacteriology,  homeostasis,  shock,  respiratory  care 
and  pulmonary  complications,  nutrition,  and  common  pharmacologic  agents 
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used  in  respiratory  care  and  anesthesia.  The  examination  was  designed 
by  the  author  and  reviewed  by  a  panel  of  three  critical  care 
specialists . 

The  examination  was  divided  into  two  sections.  Part  I  consisted  of 
99  "type  A"  questions  (only  one  possible  answer)  and  Part  II  consisted 
of  51  "type  K"  questions  (four  alternative  answers  in  one  of  five 
possible  combinations). 

Each  of  the  examinees  were  asked  to  answer  seven  demographic 
questions  (Questions  151  -  157)  concerning  professional  qualifications, 
postgraduate  education  and  background  and  the  relevancy  of  the  content 
of  the  examination  to  the  discipline  of  critical  care  medicine  (APPENDIX 
4). 

The  instructions  for  the  technique  in  marking  the  answer  sheet  were 
presented  at  the  beginning  of  each  section  and  were  reviewed  with  the 
examinees  by  an  invigilator  at  the  time  of  the  examination.  The  time 
limit  was  exactly  two  hours.  The  answer  sheets  were  scored  by  optical 
scanner . 

Logging  of  User  Performance 

The  following  data  were  automatically  recorded  for  each  user  - 
CLINICAL  REFERENCE  LIBRARY  encounter: 

1)  the  date,  start  time  and  total  elapsed  time  of  the  encounter 

2)  the  personal  identification  number  of  the  user,  and 

3)  the  modules  accessed  and  the  total  elapsed  time  on  each  module. 

Questionaires 

The  following  data  were  recorded  if  the  user  elected  to  respond  to 
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the  prompts  of  a  questionaire  (APPENDIX  5): 

1)  the  purpose  of  the  encounter  i.e.  problem-solving,  continuing 
medical  education  or  recreation 

2)  if  the  CLINICAL  REFERENCE  LIBRARY  was  accessed  for 
problem-solving,  was  the  information  present  and,  if  so,  was  it 
helpful? 

3)  the  performance  of  the  CLINICAL  REFERENCE  LIBRARY  with  respect 
to  the  library,  staffmen,  fellow  colleagues  and  attending 
seminars 

4)  a  rating  of  the  medical  content  of  the  CLINICAL  REFERENCE 
LIBRARY 

5)  a  rating  of  user  competence  before  and  after  accessing  the 
CLINICAL  REFERENCE  LIBRARY 

6)  the  degree  to  which  the  user  feels  the  CLINICAL  REFERENCE 
LIBRARY  added  to  his/her  fund  of  knowledge,  and 

7)  overall  evaluation  of  the  CLINICAL  REFERENCE  LIBRARY  and  its 
comparison  to  other  teaching  methods. 

Each  participant  answered  a  questionaire  similar  to  that  described 
above  after  writing  the  post-test  examination  (APPENDIX  6). 

ANALYSIS  OF  DATA 

An  analysis  of  variance  was  used  to  test  the  differences  of 
participant  mean  performance  on  the  basis  of  a  multiple  choice  question 
examination.  The  alpha  level  was  set  at  0.05.  This  procedure  was  used 
to  determine  intergroup  differences  for  the  pre-  and  post-tests  (i.e. 
medical  students  versus  nurses  versus  residents)  .  Using  gain  scores  as 
data  inputs  the  independent  samples  t-test  was  used  to  determine 
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differences  between  comparable  populations  in  the  experimental  and 
control  groups  (i.e.  residents  versus  residents,  nurses  versus  nurses 
and  medical  students  versus  medical  students)  for  the  pre-  and 
post-tests . 

The  questionaire  data  were  summarized  by  user  group  as  follows: 

1)  the  modules  accessed 

2)  the  reason  for  accessing  the  CLINICAL  REFERENCE  LIBRARY 

3)  the  overall  rating  of  the  CLINICAL  REFERENCE  LIBRARY 

4)  the  rating  of  the  medical  content  of  the  CLINICAL  REFERENCE 
LIBRARY 

5)  the  rating  of  user  competence  before  and  after  using 
the  CLINICAL  REFERENCE  LIBRARY 

6)  the  degree  to  which  the  user  felt  the  CLINICAL  REFERENCE 
LIBRARY  added  to  his/her  fund  of  knowledge,  and 

7)  the  overall  evaluation  of  the  CLINICAL  REFERENCE  LIBRARY  with 
respect  to  other  teaching  methods. 

The  following  logging  data  were  cross  tabulated  by  user  group: 

1)  the  modules  accessed  and  the  total  duration  of  time  spent  on 
each  module,  and 

2)  the  number  of  accesses  to  the  CLINICAL  REFERENCE  LIBRARY. 

SUMMARY 

In  this  chapter,  the  methodology  employed  to  evaluate  the  effective 
ness  of  the  CLINICAL  REFERENCE  LIBRARY  as  a  teaching  tool  was  described 
The  experimental  design  and  the  qualitative  and  quantitative  data 
gathering  techniques  were  also  described.  The  chapter  also  documents 
how  the  results  were  combined  in  the  final  analysis.  The  results  are 
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presented  in  the  next  chapter. 


CHAPTER  VII 
RESULTS 
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This  chapter  presents  the  results  of  the  quantitative  and 
qualitative  techniques  used  to  evaluate  the  CLINICAL  REFERENCE  LIBRARY. 
The  chapter  is  divided  into  four  parts.  The  first  part  is  devoted  to 
the  analysis  of  the  multiple  choice  question  examination  which  was  used 
to  determine  cognitive  gain.  The  results  of  the  computer  log  are 
presented  in  part  two.  The  results  of  the  questionaires  are  presented 
in  part  three.  The  chapter  is  summarized  in  part  four. 

INTRODUCTION 

Eighty-six  subjects  participated  in  this  study.  Participants  had 
similar  backgrounds  (i.e.  training,  experience)  in  their  respective 
subgroups.  All  participants  assigned  to  the  control  group  (16  medical 
students,  13  nurses,  and  17  residents)  completed  the  pre-  and  post-test 
examinations.  Of  the  40  participants  assigned  to  the  experimental 
group,  only  23  ultimately  finished.  Twelve  (60%)  of  the  nurses  and  5 
(42%)  of  the  residents  withdrew  from  the  study.  Reasons  for  not 
continuing  included: 

1)  too  busy  to  take  part  (eleven  nurses  and  five  residents),  and 

2)  termination  of  employment  (one  nurse). 

All  medical  students  assigned  to  the  experimental  group  completed  the 
pre-  and  post-test  examinations.  Sixty-nine  participants  were  evaluated 
prospectively  over  a  12-month  period. 

EXAMINATION  RESULTS 

Tables  2  and  3  present  the  descriptive  statistics  for  the  control 
and  experimental  groups.  The  results  are  graphically  illustrated  in 
Figure  1.  Tests  for  the  homogeneity  of  variances  across  groups  revealed 
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no  statistically  significant  difference  between  comparable  subgroups 
(i.e.  medical  students  versus  medical  students,  nurses  versus  nurses, 
and  residents  versus  residents). 


Table  2 


Results  of  the  Multiple  Choice  Question  Examinations  I 


CONTROL  GROUP 


N 

PRE. 

X 

-TEST 

S 

POST- 

X 

-TEST 

S 

MEAN 

DIFFERENCE 

SIGNIFICANCE 

LEVEL 

Medical 

Students 

16 

63.5 

12.1 

75.2 

1 1.2 

+  11.7 

p<0.05 

Nurses 

13 

47.9 

10.5 

51.9 

12.2 

4.0 

NS 

Residents 

17 

76.1 

13.4 

88.1 

13.9 

+  12.0 

p<0.05 

N 


EXPERIMENTAL 

GROUP 

PRE-TEST 

POST -TEST 

MEAN 

SIGNIFICANCE 

X  S 

X  S 

DIFFERENCE 

LEVEL 

Medical 

Students 

8 

60.8 

13.1 

91.5 

13.1 

+30.7 

Nurses 

8 

47.1 

9.4 

48.6 

9.3 

+1.5 

Residents 

7 

72.3 

10.7 

102.9 

16.8 

+30.6 

p<0. 05 
NS 


p<0. 05 
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Pre-test  Post-test 


Control  Group  Experimental  Group 

□  Medical  Students  ■ 

A  Nurses  ▲ 

o  Residents  • 

Figure  1:  Participant  Performance  on  the  Multiple 
Choice  Question  Examinations 
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With  regard  to  the  control  group,  medical  students  had  mean  scores 
of  63.5  and  75.2  (out  of  a  possible  150)  on  the  pre-  and  post-tests, 
respectively  (t=-7.78,  df=15,  p<0.05).  The  residents’  mean  score  on  the 
pre-test  was  76.1  and  rose  to  88.1  on  the  post-test  (t=-7.68,  df=16, 
p<0.05).  Nurses  had  pre-  and  post-test  mean  scores  of  47.9  and  51.9, 
respectively  (t=-1.65,  df=12,  p>0.05).  Medical  students  and  residents 
increased  their  mean  test  scores  on  the  average  of  about  12  questions 
while  the  nurses  increased  their  mean  test  scores  by  only  4  questions. 
These  results  are  summarized  in  Table  2. 

With  regard  to  the  experimental  group,  the  mean  score  for  the 
medical  students  on  the  pre-test  was  60.8  and  rose  to  91.5  on  the 
post-test  (t=-14.0,  df=7,  p<0.05).  Residents  had  a  mean  score  of  72.3 
and  102.9  on  the  pre-  and  post-tests,  respectively  ( t  =— 6 .17,  df=6, 
p<0.05).  Nurses  had  pre-  and  post-test  mean  scores  of  47.1  and  48.6, 
respectively  (t=-1.97,  df=7,  p>0.05).  Medical  students  and  residents 
increased  their  mean  test  scores  by  about  31  questions  while  the  nurses 
increased  their  mean  test  scores  by  about  2  questions.  These  results 
are  summarized  in  Table  2. 

The  pre-test  mean  scores  between  participants  in  each  subgroup 
revealed  no  statistically  significant  difference  between  the  control  and 
experimental  groups.  However,  the  difference  in  post-test  mean  scores 
between  experimental  and  control  medical  students  was  statistically 
significant  (p<0.05).  This  was  also  true  for  the  difference  in  posttest 
mean  scores  between  experimental  and  control  residents.  There  was, 
however,  no  significant  cognitive  "gain"  in  the  nurse  subgroups 
(p>0.05).  These  results  are  summarized  in  Table  3. 
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Table  3 

Results  of  the  Multiple  Choice  Question  Examinations  II 

PRE-TEST 


CONTROL 

X 

EXPERIMENTAL 

X 

MEAN 

DIFFERENCE 

SIGNIFICANCE 

LEVEL 

Medical 

Students 

63.5 

60.8 

-2.7 

NS 

Nurses 

47.9 

47.1 

-0.8 

NS 

Residents 

76.1 

72.3 

-3.8 

NS 

POST-TEST 

CONTROL 

X 

EXPERIMENTAL 

X 

MEAN 

DIFFERENCE 

SIGNIFICANCE 

LEVEL 

Medical 

Students 

75.2 

91.5 

+  16.3 

p<0.05 

Nurses 

51.9 

48.6 

-3.3 

NS 

Residents 

88.1 

102.9 

+  14.8 

p<0. 05 

LOGGING  RESULTS 

Analyses  of  the  logging  data  are  presented  in  Tables  4  and  5.  The 
CLINICAL  REFERENCE  LIBRARY  was  accessed  496  times.  A  total  of  576 
modules  were  accessed.  During  the  period  of  the  study,  (February  15  to 
August  14,  1982)  the  CLINICAL  REFERENCE  LIBRARY  was  utilized  for  a  total 


of  141.7  hours. 
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Table  5 

CLINICAL  REFERENCE  LIBRARY  Logging  Data  II 


NUMBER  OF  MODULES 
ACCESSED 

TOTAL  TIME 
(MIN) 

ELPASPED 

(HR) 

AVERAGE 
ACCESS  TIME 
(MIN) 

Medical 

Students 

218 

5012 

83.5 

23.0 

Nurses 

112 

940 

42.5 

8.4 

Residents 

246 

2552 

15.7 

10.4 

Although  the  medical  students  accessed  the  system  less  often  than 
the  residents  (218  versus  246  accesses)  they  spent  more  time  utilizing 
the  system  (average  access  time  about  23  minutes  versus  10.4  minutes). 
The  six  most  frequently  accessed  modules  were  ACID-BASE  REGULATIION, 
PULMONARY  PHYSIOLOGY,  ACUTE  POISONINGS,  INVASIVE  HEMODYNAMIC  MONITORING, 
OXYGEN  AND  CARBON  DIOXIDE  TRANSPORT  and  ENDOTRACHEAL  INTUBATION  AND 
TRACHEOSTOMY.  Almost  47%  of  the  medical  students’  time  were  spent  on 
the  basic  science  programs  (modules  #1,2, 3, 5, 6,  and  7)  (APPENDIX  2). 

The  total  time  spent  on  the  CLINICAL  REFERENCE  LIBRARY  by  the  medical 
students  was  about  83*5  hours. 

Analysis  of  the  nurse  logging  data  was  disappointing.  The  nurses 
accessed  the  CLINICAL  REFERENCE  LIBRARY  less  than  one  half  the  number  of 
times  of  either  the  residents  or  the  medical  students.  Fifty  percent  of 
the  nurses  accessed  four  or  less  modules.  The  three  most  frequently 
used  modules  were  ENDOTRACHEAL  INTUBATION,  PULMONARY  PHYSIOLOGY  and 
ACID-BASE  REGULATION.  Average  access  time  was  about  8.4  minutes.  The 
total  time  spent  <?n  the  CLINICAL  REFERENCE  LIBRARY  by  the  nurses  was 
about  15.7  hours. 

The  two  most  utilized  modules  by  the  residents  were  ACUTE  POISONINGS 
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AND  INVASIVE  HEMODYNAMIC  MONITORING.  One  resident  spent  more  than  50% 
of  his/her  time  on  ACUTE  POISONINGS.  Four  of  the  seven  residents 
accessed  more  than  1/4  of  the  modules.  The  average  access  time  was 
about  10.4  minutes.  The  total  time  spent  on  the  CLINICAL  REFERENCE 
LIBRARY  was  42.5  hours. 

RESULTS  OF  QUESTIONAIRES 

The  computer  questionaire  (APPENDIX  5)  was  answered  on  97  different 
occasions.  The  CLINICAL  REFERENCE  LIBRARY  was  accessed  496  times. 
Despite  this  poor  response,  it  was  possible  to  deduce  the  following 
generalizations : 

1)  medical  students  and  nurses  used  the  CLINICAL  REFERENCE 
LIBRARY  for  continuing  medical  education 

2)  residents  used  the  CLINICAL  REFERENCE  LIBRARY  primarily  for 
problem-solving,  and 

3)  medical  students  spent  longer  periods  of  time  at  the  terminal 
than  either  the  residents  or  nurses. 

The  remainder  of  this  section  is  devoted  to  the  analysis  of  the 
questionaire  administered  after  the  post-test  examination  (APPENDIX  6) . 
User  responses  to  this  questionaire  are  presented  in  Tables  6  to  11. 

In  general,  those  participants  who  completed  the  sequence  of  modules 
found  the  experience  rewarding.  The  most  negative  responses  or  the  lack 
of  positive  responses  were  expressed  by  the  nurses. 

Use  of  the  CLINICAL  REFERENCE  LIBRARY 

QUESTION  1  confirmed  that  medical  students  and  nurses  used  the 
CLINICAL  REFERENCE  LIBRARY  for  continuing  medication  while  residents 


71 


most  often  used  the  system  for  problem-solving.  The  results  are 
presented  in  Table  6. 


Table  6 

QUESTION  1 :  Which  of  the  following  was  the  most  common  reason  for 
accessing  the  CLINICAL  REFERENCE  LIBRARY? 

Continuing 

Problem-Solving  Medical  Education  Recreation 

Medical 

Students  -  8/8 

Nurses  -  8/8 

Residents  6/7  1/7 

User  Assessment  of  the  CLINICAL  REFERENCE  LIBRARY 

QUESTIONS  2  to  8  were  designed  in  such  a  way  that  respondents  were 
compelled  to  express  a  degree  of  agreement  or  disagreement  on  the 
subject  in  question. 

All  respondents  rated  the  CLINICAL  REFERENCE  LIBRARY  as  average  or 
above  average  with  respect  to  the  hospital  library,  consultation  with 
staffmen  or  fellow  colleagues  or  attending  seminars  (QUESTION  2). 
Nurses  were  more  likely  to  be  indifferent  whereas  residents  or  medical 
students  felt  that  the  CLINICAL  REFERENCE  LIBRARY  was  more  useful  than 
the  library,  consultation  with  fellow  colleagues  or  by  attending 
seminars.  The  results  are  presented  in  Table  7. 
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Table  7 

QUESTION  2:  How  would  you  rate  the  CLINICAL  REFERENCE  LIBRARY... 


a.  compared  to 

useless 

Medical  Students 

Nurses 

Residents 

b.  compared  to 

useless 

Medical  Students 

Nurses 

Residents 

c.  compared  to 

useless 

Medical  Students 

Nurses 

Residents 

d.  compared  to 

useless 

Medical  Students 

Nurses 

Residents 


the  library? 

1  2 

3 

4 

5 

useful 

-  - 

- 

3/8 

5/8 

-  - 

6/8 

- 

2/8 

-  - 

— 

- 

7/7 

consultation  with 

staffmen? 

1  2 

3 

4 

5 

useful 

-  - 

5/8 

- 

3/8 

-  - 

- 

1/8 

7/8 

-  - 

- 

4/7 

3/7 

consultation  with 

fellow  colleagues? 

1  2 

3 

4 

5 

useful 

-  - 

- 

- 

8/8 

-  - 

5/8 

1/8 

2/8 

-  - 

- 

- 

7/7 

attending 

seminars? 

1  2 

3 

4 

5 

usef  ul 

-  - 

- 

- 

8/8 

-  - 

5/8 

1/8 

2/8 

_ 

7/7 

Users  were  asked  in  QUESTION  3  to  rate  the  medical  content  of  the 
CLINICAL  REFERENCE  LIBRARY.  All  medical  students  and  residents  felt 


that  the  information  was  very  valuable,  current,  accurate  and  medically 
precise.  The  nurses  were  more  negative  about  the  value  of  the  CLINICAL 
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REFERENCE  LIBRARY.  The  results  are  presented  in  Table  8. 


Table  8 

QUESTION  3:  How  would  you  rate  the  medical  content  of  the  CLINICAL 
REFERENCE  LIBRARY? 


a.  worthless 

1 

2 

3 

4 

5 

valuable 

Medical  Students 

- 

- 

- 

- 

8/8 

Nurses 

- 

- 

1/8 

2/8 

5/8 

Residents 

_ 

7/7 

b.  out-of- 

1 

2 

3 

4 

5 

current 

date 

Medical  Students 

- 

- 

- 

- 

8/8 

Nurses 

- 

- 

- 

- 

8/8 

Residents 

_ 

_ 

7/7 

c.  contains 

1 

2 

3 

4 

5 

error-fre 

errors 

Medical  Students 

- 

- 

- 

- 

8/8 

Nurses 

- 

- 

- 

- 

8/8 

Residents 

_ 

_ 

_ 

7/7 

d.  medically 

1 

2 

3 

4 

5 

medically 

imprecise 
Medical  Students 

- 

- 

— 

- 

8/8 

precise 

Nurses 

- 

- 

- 

- 

8/8 

Residents 

7/7 

Most  respondents,  especially  medical  students  and  residents,  felt 
that  they  acquired  knowledge  during  their  rotation  through  the  intensive 
care  unit  and  that  such  knowledge  was  attributable  to  the  CLINICAL 
REFERENCE  LIBRARY  (QUESTIONS  4  to  6).  Two  nurses  felt  that  the  CLINICAL 
REFERENCE  LIBRARY  contributed  significantly  to  their  knowledge.  The 
other  nurses  were  not  otherwise  impressed.  The  results  are  presented  in 


Table  9. 
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Table  9 


QUESTION  4:  How  would 

you  rate 

your 

level 

of  overall 

competence  in 

critical 

care  medicine 

before 

using 

the 

CLINICAL  REFERENCE 

LIBRARY? 

average  or 

1 

2 

3 

4 

5 

outstanding 

below  average 

Medical  Students 

5/8 

3/8 

— 

- 

- 

Nurses 

5/8 

3/8 

- 

- 

- 

Residents 

1/7 

3/7 

3/7 

- 

- 

QUESTION  5:  How  would  you  rate  your  level 

of  overal 

compentence  in 

critical 

care  medicine 

after 

using 

the 

CLINICAL  REFERENCE 

LIBRARY? 

average  or 

1 

2 

3 

4 

5 

outstanding 

below  average 

Medical  Students 

- 

- 

5/8 

3/8 

- 

Nurses 

5/8 

1/8 

2/8 

- 

- 

Residents 

- 

- 

1/7 

6/7 

- 

QUESTION  6:  To  what 

degree  do 

you 

feel  that  the 

i  CLINICAL  REFERENCE 

LIBRARY 

added  to 

your 

fund  of 

knowledge 

during  your 

rotation 

through 

ICU? 

did  not  add 

1 

2 

3 

4 

5 

did  add 

Medical  Students 

- 

- 

— 

5/8 

3/8 

Nurses 

5/8 

1/8 

1/8 

1/8 

„ 

Residents 


6/7 


1/7 
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Most  participants  felt  that  the  effort  invested  in  using  the 
CLINICAL  REFERENCE  LIBRARY  was  worthwhile  (QUESTION  7).  About  75%  of 
the  nurses  were  noncommital.  The  results  are  presented  in  Table  10. 

Table  10 

QUESTION  7:  Learning  this  material  by  computer  was... 


a.  a  waste 

1 

2 

3 

4 

5 

time  well 

of  time 

Medical  Students 

— 

- 

- 

1/8 

7/8 

spend 

Nurses 

1/8 

- 

5/8 

2/8 

- 

Residents 

- 

- 

1/7 

5/7 

1/7 

b.  boring 

1 

2 

3 

4 

5 

enjoyable 

Medical  Students 

- 

- 

- 

6/8 

2/8 

Nurses 

- 

- 

6/8 

1/8 

1/8 

Residents 

- 

- 

1/7 

5/7 

1/7 

c.  unsat¬ 

1 

2 

3 

4 

5 

satisfyin* 

isfying 

Medical  Students 

- 

- 

- 

4/8 

4/8 

Nurses 

- 

- 

6/8 

1/8 

1/8 

Residents 

6/7 

1/7 

QUESTION  8  was  designed  to  determine  which  teaching  methods  the 
participants  preferred.  Nurses  preferred  consultation  with  staffmen  or 
one  of  their  peers  and  bedside  teaching.  Only  50%  of  the  nurses 
preferred  computer-presented  material.  Medical  students  and  residents 
unanimously  preferred  seminars,  bedside  teaching,  computer-presented 
material  and  discussion  with  staffmen.  The  results  are  presented  in 


Table  11. 
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SUMMARY 

In  this  chapter,  the  results  of  the  evaluative  process  of  the 
CLINICAL  REFERENCE  LIBRARY  were  presented.  Three  specific  findings 


emerged 

from  the  data: 

1) 

medical  students  and  residents  exposed  to  the  CLINICAL 

REFERENCE  LIBRARY  "acquired"  more  knowledge  than  their  control 

counterparts  using  conventional  teaching  methods 

2) 

medical  students  and  nurses  accessed  the  CLINICAL  REFERENCE 

LIBRARY  for  continuing  medical  education  whereas  residents  used 

the  system  primarily  for  problem-solving,  and 

3) 

a  general  attitudinal  ambivalence  was  directed  toward  the 

CLINICAL  REFERENCE  LIBRARY  by  the  nurses. 

The  next  chapter  is  devoted  to  discussion  of  the  results  and 
recommendations  for  further  research  and  development. 


CHAPTER  VIII 

DISCUSSION  AND  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

AND  DEVELOPMENT 
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There  have  been  no  reports  on  the  effect  of  computer-presented 
information  on  medical  knowledge  acquisition  in  the  intensive  care  unit. 
Indeed,  the  degree  of  cognitive  gain  in  any  clinical  environment 
involving  a  computer-based  information  system  has  not  been  previously 
documented.  Under  laboratory  conditions,  however,  computer-based 
education  has  contributed  to  the  acquisition  of  knowledge  (46). 

In  this  study  medical  students  and  residents  demonstrated  signif¬ 
icant  gains  in  knowledge  during  their  rotation  through  the  intensive 
care  unit.  This  result  was  not  unexpected  in  that  these  individuals  are 
training  to  be  physicians,  albeit  they  are  at  different  levels  of 
training.  One  would  also  expect  medical  students  to  be  less 
knowledgeable  than  residents.  This  was  confirmed  in  both  the  control 
and  experimental  groups  by  analysis  of  the  pre-  and  post-test  mean 
scores.  Most  importantly,  however,  medical  students  and  residents 
exposed  to  the  CLINICAL  REFERENCE  LIBRARY  demonstrated  statistically 
significant  gains  in  knowledge  when  compared  to  their  control  counter¬ 
parts  . 

These  results  do  not  mean  that  the  gains  in  knowledge  by  these  two 
groups  was  due  to  the  CLINICAL  REFERENCE  LIBRARY  per  se.  Ideally,  one 
would  hope  that  the  CLINICAL  REFERENCE  LIBRARY  was  the  source  for  their 
knowledge.  It  is  possible,  however,  that  other  factors  account  for  this 
gain : 

1)  the  information  contained  within  the  CLINICAL  REFERENCE 
LIBRARY  stimulated  further  interest,  thereby  compelling  the 
user  to  seek  more  knowledge  elsewhere 

2)  the  CLINICAL  REFERENCE  LIBRARY  reinforced  or  supplemented 
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knowledge  acquired  elsehwere  or  by  other  means,  or 

3)  the  information  contained  within  the  CLINICAL  REFERENCE  LIBRARY 
was  insufficient,  thereby  requiring  the  user  to  look  elsewhere  , 

The  role  of  the  CLINICAL  REFERENCE  LIBRARY  as  a  teaching  tool  was 
confirmed  in  part  by  the  reasons  why  the  CLINICAL  REFERENCE  LIBRARY  was 
used  by  the  various  participants.  Medical  students  and  nurses  used  the 
CLINICAL  REFERENCE  LIBRARY  for  continuing  medical  education.  The 
preference  of  medical  students  to  access  basic  science  material  for  long 
periods  supports  this.  On  the  other  hand,  residents  cited  problem¬ 
solving  as  the  most  common  reason  for  using  the  CLINICAL  REFERENCE 
LIBRARY.  Residents  tended  to  access  clinical  topics  more  often  and  for 
shorter  periods  of  time;  that  is,  they  were  probably  looking  for 
information . 

On  at  least  two  separate  occasions,  a  resident  was  known  to  access 
the  CLINICAL  REFERENCE  LIBRARY  for  the  purpose  of  gathering  information 
for  ICU  service  rounds.  The  sections  reviewed  were  methanol  poisoning 
and  ethylene  glycol  poisoning. 

A  secondary  but  equally  important  role  was  the  use  of  the  CLINICAL 
REFERENCE  LIBRARY  as  a  '’check"  on  patient  management.  Again,  the  most 
popular  module  was  ACUTE  POISONINGS.  The  clinical  problems  most  likely 
to  require  "checking"  were  acid  alcohol  poisoning  and  overdose  due  to 
tricyclic  antidepressants. 

It  is  interesting  to  note  that  the  most  frequently  utilized  modules, 
ACUTE  POISONINGS,  PULMONARY  PHYSIOLOGY,  ENDOTRACHEAL  INTUBATION  AND 
TRACHEOSTOMY,  ACID-BASE  REGULATION  and  INVASIVE  HEMODYNAMIC  MONITORING, 
are  topics  intimately  associated  with  the  management  of  the  critically 
ill  patient.  Although  many  of  the  topics  in  the  CLINICAL  REFERENCE 
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LIBRARY  pertain  to  the  practice  of  medicine  in  general,  ACUTE  POISONINGS 
and  INVASIVE  HEMODYNAMIC  MONITORING  are  topics  confined  primarily  to  the 
area  of  critical  care  medicine.  As  a  group,  the  residents  accessed 
these  two  topics  most  often.  Medical  students,  on  the  other  hand, 
preferred  ACID-BASE  REGULATION  and  PULMONARY  PHYSIOLOGY.  ENDOTRACHEAL 
INTUBATION  and  TRACHEOSTOMY  and  PULMONARY  PHYSIOLOGY  were  the  two  most 
popular  topics  with  the  nurses. 

The  participation  of  the  nurses  assigned  to  the  experimental  group 
was  disappointing.  However,  this  ambivalence  was  not  unexpected  despite 
the  author’s  attempts  to  encourage  participation.  There  are  at  least 

three  possible  explanations ' as  t©  why  the  nurses  did  not  demonstrate 
significant  gains  in  knowledge  after  using  the  CLINICAL  REFERENCE 
LIBRARY.  First,  there  is  at  this  time  a  general  ambivalence  among 
nurses,  a  factor  which  is  beyond  the  control  of  the  author.  The  nursing 
profession  is  plagued  with  labor  unrest  and  manpower  problems. 

The  second  explanation  is  based  on  the  nurses’  attitudes  toward  the 
Patient  Data  Management  System.  Even  though  the  PDMS  was  introduced  to 
the  intensive  care  unit  in  November  of  1980,  there  still  remain  strong 
negative  views  directed  toward  the  use  and  the  feasibility  of 
computerized  patient  data  management.  Nurses  have  expressed  the 
following  views  about  the  PDMS  (and  not  the  CLINICAL  REFERENCE  LIBRARY) 
to  the  author  on  numerous  occasions: 

1)  the  PDMS  takes  time  away  from  patient 

2)  the  PDMS  has  been  plagued  with  numerous  ’’bugs"  and  "down  times" 

3)  data  is  often  duplicated  by  hand  before  the  ultimate  hard  copy 
is  produced 

4)  some  nurses  lack  basic  typing  skills,  and 
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5)  the  mechanics  of  the  system  are  difficult  to  learn. 

Nurses  tend  to  resist  the  introduction  of  alteration  of  any  routine. 

This  was  most  noticeable  a  few  years  ago  with  the  introduction  of 
hemodynamic  monitoring  in  our  unit.  It  can  be  argued  that  a  negative 
attitude  directed  toward  the  PDMS  can  also  be  referred  to  the  CLINICAL 
REFERENCE  LIBRARY  merely  because  of  its  intimate  association  with  the 
PDMS. 

Thirdly,  the  lack  of  sufficient  encouragement  by  the  senior  nursing 
staff,  in  the  author’s  opinion,  was  instrumental  in  the  failure  to 
acquire  sufficient  data  from  this  subgroup.  The  Charge  Nurse  and  two 
Assistant  Charge  Nurses  were  eager  to  participate  in  the  study  and 
offered  to  act  as  a  liaison  between  the  author  and  the  other  nurses. 

All  three  withdrew  from  the  study  shortly  after  it  was  started. 

On  the  basis  of  these  findings,  the  following  suggestions  for 
further  research  are  presented.  The  CLINICAL  REFERENCE  LIBRARY,  like 
the  problem  oriented  medical  record,  must  become  an  integral  part  of  the 
intensive  care  delivery  system  before  further  attempts  are  made  to 
evaluate  its  role  and  effectiveness  in  critical  care  medicine.  A 
well-structured  unit  with  a  stable  multidisciplinary  team  with  a  true 
commitment  to  the  care  of  the  critically  ill  is  required.  The 
Director(s),  Charge  Nurse  and  Chief  Resident  must  also  have  a  genuine 
belief  in  the  concept  of  the  CLINICAL  REFERENCE  LIBRARY.  The 
responsibility  for  the  implementation  of  this  proposal  must  lie  with  the 
Director.  Then,  and  only  then,  can  a  more  representative  analysis  of 
the  CLINICAL  REFERENCE  LIBRARY  in  the  ICU  be  made. 

Underutilization  and  cost  ineffectiveness  are  detrimental  to  the 
concept  of  the  PDMS  and  the  CLINICAL  REFERENCE  LIBRARY.  Siegal 


demonstrated  a  20 %  increase  in  utilization  of  a  computer-based 
physiologic  CARE  system  by  all  personnel  after  the  introduction  of 
"living  textbook"  programs  (59).  The  effect  of  the  CLINICAL  REFERENCE 
LIBRARY  on  the  utilization  of  the  PDMS  could  be  determined  in  the 
following  manner.  The  study  would  be  undertaken  in  two  phases.  Phase 
one  would  involve  monitoring  the  use  of  the  PDMS.  The  following  data 
would  be  automatically  recorded  for  each  user-computer  encounter: 

1)  the  date,  start  time  and  total  elapsed  time 

2)  the  personal  identification  number  of  the  user,  and 

3)  the  subsystem  accessed  and  the  total  time  elapsed  on  each 
subsystem. 

In  phase  two  the  CLINICAL  REFERENCE  LIBRARY  would  be  introduced. 

Analyses  would  be  performed  for  each  group  of  participants  (i.e.  medical 
students,  nurses  and  other  paramedical  personnel  and  residents).  The 
results  would  be  tabulated  and  analyzed  for  differences  in  the  use  of 
each  of  the  subsystems  across  groups  before  and  after  the  introduction 
of  the  CLINICAL  REFERENCE  LIBRARY. 

Having  demonstrated  that  medical  students  and  residents  acquire 
knowledge  after  exposure  to  an  ICU  computer-based  clinical  information 
system,  the  next  step  in  user  evaluation  should  include  testing  for 
proper  patient  management.  For  the  purpose  of  clarity,  proper  patient 
management  encompasses  the  institution  of  appropriate  investigations, 
the  development  of  diagnoses  and  the  implementation  of  appropriate 
therapy.  Testing  for  proper  patient  management  could  be  accomplished  by 
repeating  the  study  described  herein  and  using  one  or  more  of  the 
following  test  instruments: 

1)  more  sophisticated  multiple  choice  questions 
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2)  clinical  scenarios  requiring  generation  and  rank  ordering  of 
investigations,  differential  diagnoses,  or  treatments,  and 

3)  sophisticated  computerized  clinical  scenarios  requiring  natural 
language  input  mode  and  which  would  require  the  generation  and 
rank  ordering  of  investigations,  differential  diagnoses  and 
treatments . 

A  fourth  and  more  sophisticated  test  would  involve  reviewing  patients' 
charts  and  comparing  what  was  reportedly  done  and/or  documented  about  a 
clinical  problem  with  a  set  of  explicit  criteria  previously  established 
by  a  group  of  specialists  in  critical  care  medicine.  The  minimum 
criteria  required  for  the  management  of  twelve  problem  areas  within 
critical  care  medicine  would  be  defined  by  a  panel  of  critical  care 
specialists.  Topics  or  problem  areas  would  include  pulmonary  embolism, 
aspiration  pneumonia,  septic  shock,  ARDS,  acid  alcohol  poisoning, 
tricyclic  antidepressant  poisoning,  salicylate  poisoning,  barbiturate 
overdose,  inhalation  injury,  hypothermia,  cerebral  edema  and 
hyperalimentation.  Baseline  measurements  of  patient  management  for  each 
group  would  involve  scrutinizing  patients'  charts  for  the  documentation 
and  management  of  clinical  problems  and  comparing  these  components  of 
patient  care  with  the  criteria  previously  established.  Patient 
management  by  the  experimental  group  would  be  examined  by  reviewing  the 
CLINICAL  REFERENCE  LIBRARY  log,  the  computerized  questionaire ,  and 
patients'  charts.  Objective  testing  for  patient  management  by  the 
control  group  would  involve  reviewing  patients'  charts  for  the 
documentation  and  management  of  clinical  problems.  On  the  basis  of 
these  reviews,  patient  management  would  be  analyzed  and  classified  as 
superior,  adequate  or  indeterminant.  For  example,  in  order  for  the 
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management  of  a  patient  with  acid  alcohol  poisoning  to  be  classified  as 
adequate  the  patient  must  receive  the  following  therapy: 

1)  support  of  respiratory  failure  and  shock 

2)  correction  of  the  metabolic  acidosis 

3)  immediate  hemodialysis,  and 

4)  administration  of  alcohol. 

For  such  an  episode  to  be  classified  as  superior,  the  following  aspects 
of  patient  management  must  be  performed: 

1)  documentation  of  serum  acid  alcohol  levels 

2)  documentation  of  the  urinalysis  for  oxalate  crystals 

3)  documentation  of  the  fundoscopic  examination 

4)  documentation  of  the  serum  calcium  and  correction  hypocalcemia, 
and 

5)  administration  of  thiamine  and  pyridoxine  supplements  (if 
ethylene  glycol  poisoning). 

Both  groups  would  be  required  to  document  all  aspects  of  patient  care  by 
using  the  problem  oriented  medical  record.  Data  on  patient  management 
of  the  two  groups  would  then  be  tabulated  and  analyzed  to  determine  if  a 
computerized  clinical  information  system  affects  clinicians'  abilities 
to  provide  patient  management. 

The  CLINICAL  REFERENCE  LIBRARY  should  be  expanded  to  include  topics 
such  as  ventilators,  the  metabolic  response  to  trauma,  blood  component 
therapy,  wound  healing,  acute  renal  failure,  dialysis,  cardiac 
tamponade,  fat  embolism  syndrome  and  the  management  of  trauma. 

Lastly,  a  literature  review  committee  should  be  set  up  to  regularly 
update  the  information  and  the  references  of  the  clinical  information 
modules.  The  primary  goal  of  this  committee  should  be  to  provide  all 
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users  of  the  CLINICAL  REFERENCE  LIBRARY  with  current  and  relevant 
information  and  literature.  This  committee  should  be  composed  of 
respected  medical  authorities  and  should  include  at  least  one  individual 
who  has  an  intimate  knowledge  about  the  operation  and  philosophy  of  the 
CLINICAL  REFERENCE  LIBRARY.  Selection  criteria  for  the  reference 
material  should  include: 

1)  state  of  the  art  review  articles 

2)  case  studies  with  substantial  reviews  of  the  literature  about  a 
specific  problem 

3)  articles  about  instrumentation  and/or  methodology  if  relevant 
clinical  applications  are  explicitly  described 

4)  editorials  which  discuss  and  evaluate  an  article  selected  from 
elsewhere  in  the  journal 

5)  articles  which  compare  treatment  modalities  which  have  direct 
clinical  application,  and 

6)  articles  which  compare  drug  mechanisms,  side  effects, 
contraindications,  dosage,  and  drug  interaction. 

SUMMARY 

The  effects  of  the  CLINICAL  REFERENCE  LIBRARY  in  the  intensive  care 
unit  were  discussed.  Recommendations  for  further  research  included: 

1)  integration  of  the  CLINICAL  REFERENCE  LIBRARY  into  the 
organization  of  the  intensive  care  delivery  system 

2)  evaluation  of  the  user’s  ability  to  properly  manage  patients 

3)  expansion  of  the  content  of  the  CLINICAL  REFERENCE  LIBRARY,  and 

4)  the  formation  of  a  review  committee  to  update  the  clinical 


information  modules. 
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APPENDIX  4 


DEMOGRAPHIC  QUESTIONAIRE  FROM  THE  MULTIPLE  CHOICE 
QUESTION  EXAMINATION 

151.  HOW  LONG  HAVE  YOU  BEEN  ACTIVELY  INVOLVED  IN  CRITICAL  CARE 
MEDICINE? 


A. 

NEVER 

B. 

LESS  THAN  3  MONTHS 

C. 

3  TO  6  MONTHS 

D. 

7  TO  12  MONTHS 

E. 

MORE  THAN  ONE  YEAR 

152.  YOUR 

PRIMARY  PROFESSION  IS: 

A. 

MEDICAL  STUDENT  (ANSWER  QUESTION  157) 

B. 

NURSE  (ANSWER  QUESTIONS  153  AND' 157) 

C. 

PHYSICIAN  (ANSWER  QUESTIONS  154  TO  157) 

D. 

OTHER  (ANSWER  QUESTION  157) 

153.  YOUR 

LEVEL  OF  NURSING  TRAINING  IS: 

A. 

CRITICAL  CARE  NURSE 

B. 

BACHELOR  OF  SCIENCE  (NURSING) 

C. 

REGISTERED  NURSE 

154.  YOUR 

PRIMARY  MEDICAL  SPECIALTY  IS: 

A. 

ANESTHESIA 

B. 

INTERNAL  MEDICINE 

C. 

SURGERY 

D. 

PEDIATRICS 

E. 

OTHER 
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155.  YOUR  LEVEL  OF  POSTGRADUATE  TRAINING  IS: 

A.  RESIDENT  I 

B.  RESIDENT  II 

C.  RESIDENT  III 

D.  RESIDENT  IV 

E.  RESIDENT  V 

156  YOU  HAVE  PREVIOUSLY  ROTATED  THROUGH: 

A.  ANESTHESIA 

B.  CORONARY  CARE  UNIT 

C.  ADULT  ICU 

D.  NICU 

E.  CARDIOVASCULAR  RECOVERY  ROOM 

157.  HOW  MANY  OF  THE  150  QUESTIONS  ON  THIS  EXAMINATION  DO  YOU 

THINK  ARE  PERTINENT  TO  THE  FIELD  OF  CRITICAL  CARE  MEDICINE? 

A.  1  TO  30 

B.  31  TO  60 

C.  61  TO  90 

D.  91  TO  120 

E.  121  TO  150 


APPENDIX  5 


COMPUTER  QUESTIONAIRE 


1.  Why  did  you  access  this  program? 

a.  problem-solving  ie.  problem  with  management  of  a 
patient 

b.  continuing  medical  education 

c.  recreation 

2.  Assuming  that  you  accessed  this  program  for  problem-solving 
reasons : 

a.  did  you  find  the  information  you  were  seeking? 

b.  did  the  information  help  resolve  a  problem  or  problems? 

3.  How  would  you  rate  the  CLINICAL  REFERENCE  LIBRARY? 
a.  compared  to  the  library 


useless 


12  3  4  5 


very  useful 


b.  compared  to  consultation  with  staffmen 


useless 


1  2  3  4  5 


very  useful 


c.  compared  to  consultation  with  fellow  colleagues 


useless 


1  2  3  4  5 


very  useful 


d.  compared  to  attending  seminars 


useless 


1  2  3  4  5 


very  useful 
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4.  How  would  you  rate  medical  content  of  CLINICAL  REFERENCE 
LIBRARY? 


5. 


a. 

Worthless 

1  2 

3 

4 

5 

valuable 

b. 

out-of-date 

1  2 

3 

4 

5 

current 

c . 

contains  errors 

1  2 

3 

4 

5 

error-free 

d. 

medically 

1  2 

3 

4 

5 

medically 

imprecise 

precise 

How  would  you  rate  your  level  of  overall  competence  in 
critical  care  medicine  before  using  the  CLINICAL  REFERENCE 


LIBRARY? 


average  or  12345  outstanding 

below  average 

6.  How  would  you  rate  your  level  of  overall  competence  in 

critical  care  medicine  after  using  the  CLINICAL  REFERENCE 
LIBRARY? 


average  or  12345  outstanding 

below  average 

7.  To  what  degree  do  you  feel  that  the  CLINICAL  REFERENCE 
LIBRARY  added  to  your  fund  of  knowledge  during  your  rotation 
through  the  ICU? 

did  not  add  12345  did  add 

8.  Learning  this  material  by  computer  was: 

a.  a  waste  of  12345  time  well  spent 

time 

b.  boring  12345  enjoyable 

c.  unsatisfying  12345  satisfying 
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Which  of 

the  following  teaching  methods 

do  you 

prefer? 

a . 

computer-presented  material 

Yes 

No 

b. 

seminars 

Yes 

No 

c. 

reading  textbooks 

Yes 

No 

d. 

reading  journals 

Yes 

No 

e . 

discussion  with  staffmen 

Yes 

No 

f . 

discussion  with  colleagues 

Yes 

No 

g. 

bedside  teaching 

Yes 

No 

APPENDIX  6 


THE  POST-TEST  EXAMINATION  QUESTIONAIRE 


1 .  Which  of  the  following  was  the  most  common  reason  for 
accessing  the  CLINICAL  REFERENCE  LIBRARY? 

a.  problem-solving  ie.  problem  with  the  management  of  a 
patient 

b.  continuing  medical  education 

c.  recreation 


2.  How  would  you  rate  the  CLINICAL  REFERENCE  LIBRARY . 

a.  compared  to  the  library? 

useless  12345  very  useful 

b.  compared  to  consultation  with  staffmen? 

useless  12345  very  useful 

c.  compared  to  consultation  with  fellow  colleagues? 

useless  12345  very  useful 

d.  compared  to  attending  seminars? 

useless  12345  very  useful 

3.  How  would  you  rate  the  medical  content  of  CLINICAL  REFERENCE 
LIBRARY? 


a.  worthless 

b.  out-of-date 

c.  contains  errors 

d.  medically 
imprecise 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


valuable 

current 

error-free 

medically 

precise 
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4.  How  would  you  rate  your  level  of  overall  competence  in 
critical  care  medicine  before  using  the  CLINICAL  REFERENCE 
LIBRARY? 

average  or  12345  outstanding 

below  average 

5.  How  would  you  rate  your  level  of  overall  competence  in 
critical  care  medicine  after  using  the  CLINICAL  REFERENCE 
LIBRARY? 

average  or  12345  outstanding 

below  average 

6.  To  what  degree  do  you  feel  that  the  CLINICAL  REFERENCE 
LIBRARY  added  to  your  fund  of  knowledge  during  your  rotation 
through  the  ICU? 

did  not  add  12345  did  add 

7.  Learning  this  material  by  computer  was... 


a . 

a  waste  of  1 

2  3  4  5 

time 

well  spent 

time 

b. 

boring  1 

2  3  4  5 

enjoyable 

c. 

unsatisfying  1 

2  3  4  5 

satisfying 

Which  of  the  following 

teaching 

methods  do 

you  prefer 

a . 

computer-presented 

material 

Yes 

No 

b . 

seminars 

Yes 

No 

c. 

reading  textbooks 

Yes 

No 

d. 

reading  journals 

Yes 

No 

e . 

discussion  with  staffmen 

Yes 

No 

f . 

discussion  with  colleagues 

Yes 

No 

g. 

bedside  teaching 

Yes 

No 

